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20.  ABSTRACT  (Contlnur  on  revetee  tide  U  neceeemt y  end  Idenlrty  by  block  number) 

^The  literature  on  performance  objectives  is  reviewed  under  the  following 
headings:  definitions  of  performance  objectives;  function  of  performance 
objectives;  pros  and  cons  of  performance  objectives;  and  form  of  performance 
objectives.  The  review  indicates  a  general  agreement,  unsupported  by 
research,  that  performance  objectives  have  value  for  learners,  instructors, 
and  designers;  that  there  is  no  operational  definition  of  the  term 
performance  objective;  and  that,  although  there  is  general  agreement  *— — y 
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regarding  the  throe  formal  characteristics  of  a  performance  objective, 
examples  of  performance  objectives  still  differ  greatly  from  one  another  in 
form.  The  results  of  six  original  studies  are  reported.^ 

Study  1  supplied  us  with  normative  data  regarding  the  "observability" 
of  99  frequently  recommended  verbs,  a  replication  of  Deno  and  Jenkins  (1969). 
The  results  confirmed  their  findings,  but  carried  with  them,  as  had  previous 
studies,  the  implication  that  the  verb  is  the  only  part  of  the  objective  worth 
considering.  Study  2  began  an  examination  of  the  effect  of  other  parts  of 
the  objecttve>  i.e.,  conditions  and/or  criteria,  on  the  already  established 
"observability"  of  the  verbs  in  Study  1.  Both  components  were  found  to  affect 
the  nature  of  the  objective  as  well  as  the  perceived  precision  of  the  verb, 
often  in  a  highly  significant  manner.  Study  3  further  developed  the  role  of 
the  three  components  of  performance  objectives.  These  data  suggested  that  the 
observed  precision  of  an  objective  results  from  an  interaction  of  all  three 
parts,  and  that  analyses  of  the  components  in  isolation  are  meaningless. 

The  search  for  unaccounted-for  variance  in  the  overall  observability  of 
objectives  led  to  an  examination  of  yet  another  segment,  the  direct  object. 

The  objectives  employed  in  Studies  1  through  3  used  an  abstract  direct  object 
("x"  or  "y")  in  order  to  minimize  differential  effects.  In  Study  4,  direct 
objects  varying  in  judged  degree  of  specificity  were  incorporated  into  the 
objectives  and  tested  for  their  specific  and  general  effects.  The  direct 
objects  were  found  to  contribute  important  additional  information  to  the 
reader  in  terms  of  both  observability  and  precision,  following  the  linear 
relation  anticipated  by  the  gradations  in  specificity.  Interestingly, 
comparisons  of  objectives  with  either  abstract  or  concrete  direct  objects 
produced  no  overall  significant  difference,  suggesting  that  readers  had 
naturally  substituted  concrete  modifiers  for  the  abstract  direct  objects  in 
order  to  "complete"  the  objective.  An  acceptable  performance  objective, 
therefore,  should  contain  not  only  a  verb  denoting  an  observable  behavior, 
but  also  appropriate  conditions,  criteria,  and  clearly  stated  or  easily 
inferred  direct  objects.  The  fifth  study  represented  an  attempt  to  determine 
whether  or  not  the  previous  results  could  be  extended  to  a  specific  target 
population  —  that  of  military  personnel  (both  trainees  and  trainers).  Quite 
simply,  the  similarity  of  the  military  personnel  responses  to  those  of  the 
earlier  studies  (collage  students)  on  the  verb  rating  tasks  strongly  supportei 
the  generalization  of  conclusions.  Further  information  on  similarity  of 
interactive  effects  will  be  required  to  provide  a  blanket  acceptance  of 
transferability.  The  final  study  in  this  report  constituted  yet  another  ill- 
fated  attempt  to  surface  an  observable  function  of  objectives.  As  Illich 
(1973)  so  tactfully  warned,  "...alchemists  failed  no  matter  how  often  they 
tried,  but  each  time  their  ’science'  yielded  new  reasons  for  their  failure, 
and  they  tried  again,"  Indeed,  through  the  pale  of  nonsignificance,  a  hopefu 
glimmer  did  occur.  The  observers  in  this  study  were  asked  to  pass  judgment  o 
the  "clarity"  with  which  the  teacher  disseminated  the  information.  Whil:  one 
supposes  that  teachers  employing  objectives  would  behave  with  corresponding 
precision,  such  was  not  the  case.  Nevertheless,  it  was  discovered  that  the- 
trained  observers  could  effectively  identify  those  groups  which  would  more 
likely  achieve.  The  presence  of  performance  objectives  appears  less  importan 
than  the  behavior  assumed  to  be  associated  with  the  use  of  objectives.  While 
this  finding  is  by  no  means  surprising,  it  may  suggest  that  reasearchers 
should  abandon  the  mehtods  currently  in  vogue  in  studies  designed  to  ascertai 
the  function  of  objectives  in  instruction.  It  is  unlikely  that  any  study  wil 
profoundly  alter  the  basic  teaching  behaviors  of  the  subjects. 
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Mair.tairine  readiness  througn  trailing  is  the  single  most  important 
tarv  peacetime  activity--  r-d  one  of  tine  most  expensive.  Today  ovpt 


half  the  defense  budget  is  for  ' people"  costs,  of  wh 
portion  is  related  to  training. 


ch  a  substantial 


To  i.n.ure  trot  trailing  ore  grams  are  cost-effec;i  ve ,  all  the  military 
services  now  use  the  systems  ap;.  roacn  to  instruction!!  pro  cram  development. 
Guidelines  for  folk-wing  the  systems  approach,  or,  a-  it  is  now  nearly 
universally  referred  to  the  Instructional  Systems  Development  process, 
are  provided  In  several  service  manuals,  pamphlets*  and  training  packages. 
Without  exception,  all  of  these  are  " object; ves -based."  i,e.s  they  proceed 
from  the  assumption  that  both  good  traini  ig  and  gaod  assessment  (or 
evaluation)  must  be  based  on  well-stated  performance  objectives. 

Nor  is  it  military  training  alone  which  ascribes  this  crucial  role* 
to  performance  objectives.  Both  business  and  education  have  done  sc,  too, 
In  these  areas,  the  tern;  behavioral  objective  is  used  more  frequently 
than  the  more  fanil^r  performance  objective.  Performance  objectives 
provide  a  basis  for  designing  curriculum,  for  developing  training  materials 
otic  procedures,  ,.nd  for  generating  tests  or  performance  measures.  They 
have  been  used  both  to  tell  trainees  what  Is  expected  of  th..-m  and  to  judge 
whether  or  not  trainers  (or  training  systems)  are  accomplishing  what  is 
expected. 

Despite  the  almost  universal  acceptance  of  the  concept  of  performance 
objectives,  there  remains  a  wide  range  of  viewpoints  concerning  both 


what  they  are  (form)  and  what  purpose  they  serve  (function).  There  is 


'-oma  rk  so '  y  1  i  tt  1  e  em; 
research  the?  a  is  "is 


•-’r.al  r*  search  *'epo**ted  in  tins  area  and  what 
tit*  > 'Video  ngir'^ica u  support  ftr  the  use  of 


pe'formar.'Ce  objective:  fo**  a-v  purpose-  Thr-  seme  auestions  which 
Ausube’  (1963)  raised  concerning  discovery  earning  apply  to  the  current 
performance  objectives  "bandwegon.  '  He  Dot  ited  c:>  that  a  careful 

review  of  the  literature  yields  three  conclusions,  which,  in  view  of  the 
wide  acceptance  of  discovery  learning,  are  most  disheartening.  The  three 
applied  to  the  area  of  performance  objectives,  ore  these:  First,  most 
of  the  articles  commonly  cited  as  reporting  results  supportive  of 
performance  objectives  actually  report,  no  research  findings  whatsoever 
but  consist  mainly  of  theoretical  discussions,  assertion,  and  conjee*. re; 
cf  descriptions  of  existing  projects  or  sy'tems  utilizing  performan u 
objectives;  and  of  enthusiastic  but  wholly  subjective  testimonials  regarding 
the  efficacy  of  performance  object! .fj.  Second,  most  of  the  reas<iably 
well -control led  studies  report  findings  winch  fail  to  support  th  contention 
that  the  performance  objective  is  an  effective  independent  vari'ble. 

Finally,  most  studies  reporting  positive  findings  either  fail  .0  control 
other  significant  variables  or  employ  questionable  technique?  of  statistical 
analysis.  Thus,  actual  examination  cf  the  research  literatue  that 
allegedly  supports  performance  objectives  reveals  that  vali‘  evidence  is 
virtually  nonexistent.  It  appears  that  the  various  enthus  asts  for  performance 
objectives  have  been  supporting  each  other  research-wise  t/  taking  in  each 
other's  laundry,  so  to  speak,  that  is,  by  citing  each  other's  opinions 
and  assertions  as  evidence  and  by  generalizing  ^-nthusi ustlcally  (if  not 
wildly)  from  equivocal  and  even  negative  findings. 
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t r -  Ucempts  to  define  performance  objectives;  the'i  w?  shal "  conriccr 
the  fc-mn  of  performance  objectives;  f '■  rial  1  / -  we  shat]  study  the  functions 
of  performance  objectives , 

Otf'nitions  of  Performance  Objectives 

Basicai1),  all  writers  refer  to  the  performance  objectiv'  as  a 
statement  describing  a  final  behavioral  state  of  a  learner.  If  learning 
Is  considered,  qu  te  broadly,  is  a  relatively  permanent  change  in  a 
learner’?  behavior  following  instruction  (Gerlach  &  E>y,  1980),  then  a 
performance  objective  is  nothing  more  or  less  than  a  statement  of  what 
someone  should  learn  or  is  learning.  Whether  such  statements  are 
referred  to  as  objectives,  aims,  goals,  intents,  or  outcomes,  they  must 
refer  to  the  intended  change  one  wishes  to  produce  in  a  learner  (Popham, 

1969).  Both  Bloom  et  al .  (1956)  and  Mager  (1962)  define  performance  objectives 

in  terms  of  change  1r»  a  learner. 

Frequently  a  performance  objective  is  closely  associated  with 
subject  matter.  A  student  pilot  learning  basic  navigation  cannot  acquire 
the  required  skill  without  some  degree  of  proficiency  in  meteorology, 

A  performance  objective  in  this  context  must  specify  not  only  the  terminal 
behavior,  but  also  the  particular  aspects  of  the  subject  matter  to  which 
the  learner  must  address  himself  in  order  that  the  desired  learning  may 
occur  (Gagni  end  Briggs,  1979). 

Some  writers  hove  attempted  to  use  operationisro  as  a  guide  for 
constructing  performance  objectives.  Operational  definitions  have  done 
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fr.ucn  to  reduce  ambiguity  in  the  language  of  the  hard  sciences.  7ief<'>an 
(1377)  asserted  that  perfo nuance  objectives  are  abstract  concepts  defined 
in  berms  of  a  rvmbe;'  of  instances  drawn  from  the  cognitive,  affective, 
cr  psychemotor  domains.  Theoretically,  this  kiro  of  objective  should 
yield  excellent  results.  Practically,  however,  it  founders  on  the  shoal 
of  precisely  those  flues Mons  which  plague  "hard"  scientists:  What  exactly 
counts  as  an  operation?  What  nappans  to  the  concepts  when  we  are  not 
performing  operations  or  if  we  have  not  yet  learned  hw  to  perrorm  them 
(MacDonald -Ross,  1973)?  Hempel  (1965)  pointed  out  that  activities,  events, 
and  attitudes  that  are  not  ascertainable  by  direct  observation  have  an 

important  and  valid  place  ir  the  educational  system.  For  instance,  in 
the  fine  arts  it  is  extremely  difficult  to  specify  an  observable  behavior 

when  judgment,  feeling,  and  creativity  play  a  major  role.  Similarly, 

when  we  think  of  pilot  training,  we  quickly  encounter  analogous  problems: 
how  does  one  write  a  performance  objective  in  operational  terms  to  deal 
with  such  elusive,  but  nevertheless  crucial,  factors  as  pilot  judgment  or 
pilot  feeling?  Perhaps,  as  MacOonald-Ross  asserted  when  speaking  of 
subjects  such  as  art,  there  arc  no  ultimate  goals  to  ba  reached,  but 
rather  standards  or  judgment  to  be  developed;  these  kinds  of  broad  goals 
represent  a  type  of  behavior  which,  since  it  is  internal,  is  not  observable 
and  consequently  not  highly  amenable  to  the  constraints  which  operationism 
Imposes  on  those  who  would  define  performance  objectives.  Similar  concerns 
In  areas  dealing  with  originality,  self-discovery,  and  creativity  have 
been  voiced  by  Eisner  (1967)  and  Burns  (1972).  Frequently,  the  only 
evidence  concerning  judgment  is  the  failure  to  make  certain  kinds  of 
mistakes  1 


o‘  ?e •  f'orr.:a''.(.e  Tojec'i  r-:s 


•  rr.  tr  fcn-r  Systems  Design  manuals  dea’  th  t'.-ee  categories  of 
functions  of  ptr“orrr?-'a  object.: ves:  (1)  an  aid  to  tie  design  c”  efficient 
•instruction  a*  c»“  tra''riirg  systems;  (?)  an  aid  to  ■  /a '»<at ng  instruction, 

( 3  an  aid  to  the  studn*  t. 

Design.  Objectives  provide  a  pent  of  departure  for  pl.irr.mg  an 
instruct : oral  on  gram  o*  a  training  system.  When  der i gning  a  system  nr 
program,  one  needs  to  know  what  a  successful  solution  will  look  like  as 
wel '  as  what  criteria  must  be  satisfied.  Performance  objectives  may 
provide  the  only  possible  rational  basis  for  evaluating  toe  success  of 
the  learning  exper'er.ce.  The  system  or  program  is  s access ful  only  when 
students  can  demonstrate  satisfactorily  what  the  objectives  predicted, 
if  these  assumptions  cf  MacPonald-Ross  (1973'  are  valid,  then  performance 
objectives  can  oe  used  as  guidelines  for  the  design  of  instruction. 

A  systematic  procedure  for  developing  ob jective-basod  instruction 
has  heen  developed  by  Gagne  and  Sriggs  (1979).  They  stated  that  when 
objectives  are  known,  on?  is  able  to  infer  what  kind  of  learned  capability 
is  being  acquiree  and  what  conditions  will  be  needed  to  bring  about  tne 
learning  wUh  greatest  efficiency.  Clearly,  then,  the  systematic  design 
cf  lessons  whicn  make  up  courses  will  result  in  the  development  of  a 
sueaale  collection  of  statements  of  Objectives.  Higher  level  objective, 
wi.l  be  formulated  which  will  depend  on  the  acquisition  of  lower  level 
objectives  or  prerequisite  skills.  The  identification  of  perfoenance 
objectives  makes  possible  the  classification  of  capabilities  into  useful 
categories..  Without  these  categories,  we  can  deal  vitn  learning  principles 


or  1 y  on  a  very  geieral  basis.  With  them,  it  becomes  possible  to  infer 
whan  kincs  of  learned  capabilities  are  being  acquired  at  any  given  point 
in  the  learning  process.  One  can  also  determine  under  what  conditions 
(internal  as  well  as  external)  the  learning  experience  takes  place. 

The  model  developed  by  Gagne  and  Briggs  is  tied  closely  to  Gagne's 
brand  of  educational  psychology.  The  hierarchy  (problen  solving,  rule 
learning,  concept  learning,  discrimination,  etc.)  has  been  described  In 
his  Conditions  of  Learning  (1977) .  It  seems  quite  unlikely  that  one  can 
apply  the  Gagne  and  Briggs  model  to  design  problems  without  some  knowledge 
of  Gagne's  more  theoretical  work.  A  simpler  model ,  also  objective-based, 
is  that  of  Dick  and  Carey  (1978).  They  clearly  state  that  the  first  step 
in  building  a  module  of  Instruction  is  to  state  the  instructional  goal, 
a  term  which  Is  nearly,  if  not  exactly,  synonymous  with  performance 
objective.  They  use  the  goal  statement  as  the  basis  for  conducting  an 
Instructional  analysis,  for  specifying  entry  behaviors,  for  generating 
test  items,  and  for  developing  instructional  strategies  and  materials.  The 
Dick  and  Carey  model,  while  simpler  to  apply  than  the  Gagne  and  Briggs 
model,  is  by  no  means  a  simplistic  one. 

While  there  are  many  other  objective-based  models  for  instructional 
design,  essentially  all  of  them  emphasize  the  same  components  as  do  the 
two  cited.  Indeed,  two  hundred  years  before  the  advent  of  performance 
objectives,  educators  (e.g.,  Herbart,  Pestalozzi)  had  pointed  out  that 
good  teaching  consists  of  determining  one's  goal,  adopting  a  course  of 
action  which  leads  to  that  goal,  and  determining  whether  or  not  the  goal 
has  been  attained.  Performance  objectives  have  not  changed  this  model; 


they  may  have  Increased  Us  r'wer  by  bringing  greater  precuion  to  the 
first  element.  Mager  (1362)  certainly  thought  so:  N-  described  the 
three  elements  in  these  terms: 

(1)  Where  an  I  gjing?,  which  refers  to  hew  to  achieve  the  objfctives; 
(2}  How  will  I  get  there?,  which  refers  to  how  to  achieve  th- 
objectlves;  arid  [3}  How  will  I  know  when  I've  arrived?,  whi<r.  is  the 
evaluation  process  of  determining  whether  or  not  the  student  has 
satisfactorily  achieved  the  objective. 

Note  well  that  the  word  objective  Is  the  ordinate  word  in  both  Statement 

2  anc  3! 

Emphasis  has  been  placed  on  the  performance  as  the  nolnt  of  departure. 
A  number  of  Instructional  designers  have  protested  that  this  is  too 
restricted  a  view  of  the  design  process,  that  the  design  prccess  can  move 
ahead  even  with  only  imperfectly  formulated  performance  objectives. 
MacDonald-Ross  ('973)  suggested  that  the  designer  should  do  his  best  in 
developing  objectives,  then  move  on  to  the  development  of  end-of-unit 
tests,  and  then  select  and  develop  the  instruction.il  materials.  This 
procedure  would  be  carried  out  until  each  component  has  been  specified 
as  clearly  as  possible.  At  this  point  one  would  develop  a  first  draft; 
quite  frequently  there  will  be  changes  In  some  of  the  objectives.  But 
of  course  one  would  be  doing  this  on  the  basis  of  evidence,  rather  than 
on  the  basis  of  seme  vaguely  conceived  or  haphazard  scheme.  It  is  well 
understood  that  although  objectives  are  intended  to  be  a  basis  for 
prescribing  course  structure  and  evaluation,  frequent  adjustments  must 
be  made  In  practice. 

Not  all  instructional  technologists  are  agreed  that  performance 
objectives  are  an  unmitigated  blessing  for  the  instructional  designer. 
Baker  (1974),  for  example,  observed  that  the  use  of  behavioral  objectives 


a 

^orrriii  *  a  h;sis  for  restructuring  instructional  programs  may  have  some 
negative  consequences.  Because  objectives  are  stated  in  operational 
language,  they  appear  to  be  mo~e  teachabU,  -Objective;  may  look  achievable 
if  they  follow  the  formula:  ’Given.,.,  the  student,  will  be  able  to..,", 
but  sucii  is  not  alv.iys  the  case.  Because  it  is  easy  to  transform  goals 
into  the  accepted  behavioral  objectives  format,  examples  of  learning  may 

f 

be  casually  produced.  Baker  stated  that  many  supervisors  and  curriculum 
specialists  feel  that  as  long  as  the  behavioral  verb  has  been  supplied, 
there  is  little  to  criticize.  She  also  questioned  whether  most  behavioral 
objectives  present  sufficient  cues  regarding  how  a  taacher  should  alter 
instruction  in  order  to  facilitate  improved  learning. 

Objectives  help  as  a  stimulus  to  clear  thinking  by  forcing  teachers 
to  think  in  specific  terms  rather  than  In  vague  ambiguities.  MacDonald- 
Ross  i!9’3)  felt  that  this  is  a  prerequisite  for  any  system  of  design  or 
planning  and  that  such  thinking  yields  the  additional  benefit  of  revealing 
value  judgments  th<*t  might  otherwise  remain  concealed.  Cnee  externalized, 
such  thinking  can  be  subjected  to  criticism  and  testing,  and  thus  instruction 
can  be  Improved.  If  objectives  can  provide  a  stimulus  for  clear  thinking, 
they  ought  to  have  the  potential  of  helping  teachers  develop  instructional 
strategies  and  methods. 

Evaluation.  Since  performance  objectives  are  descriptions  of  what 
must  be  observed  In  order  to  verify  that  learning  has  occurred,  they  should 
provide  a  basis  for  the  development  of  tests  or  assessment  procedures, 
both  for  determining  learner  progress  and  for  assessing  the  quality  of 
the  instructional  system.  Objective-based  assessment  need  not  occur  only 
at  the  completion  cf  *  progran  or  unit  Briggs  (1970)  stated  that 
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objectives  are  useful  for  determining  whether  or  not  students  need 
remedial  work.  Obviously,  tes~.s  on  specific  objectives  serve  as  a  guide 
for  determining  whether  the  student  is  ready  to  go  on  to  the  next 
objective;  tests  for  units  of  instruction  can  reveal  the  learner's 
mastery  of  more  complex  objectives  or  of  clusters  of  objectives;  end-of- 
course  tests  can  indicate  the  students'  ability  to  solve  we  complex 
problems  or  to  apply  their  knowledge  to  a  wider  range  of  situations. 
Objective-based  tests  can  also  be  used  prior  to  instruction  to  help  in 
identifying  students  who  lack  the  orerequisltes  needed  to  succeed  (Gagne 
and  Briggs,  1979)  as  well  as  to  identify  those  who  need  no  Instruction 
because  they've  already  attained  the  objective. 

Aid  to  students.  When  behavioral  objectives  are  given  to  a  student 
prior  to  instruction,  they  may  provide  guidance  to  the  student  In 
processing  information.  Delerline  (1968)  said  that  if  students  are  told 
precisely  what  the  objectives  are.  In  the  form  of  mlnititm  performance 
requirements,  and  if  they  are  given  sample  test  questions,  performance 
car  be  improved.  If  behavioral  objectives  are  used  to  tell  the  student 
exactly  how  he  is  going  to  be  tested,  anxiety  may  be  diminished. 

Performance  objectives  may  <  n  direct  students'  learning  {Duchastel 
and  Merrill,  1973;  <apfer,  1970;  Kibler  et  al.,  1974),  since  such 
objectives  may  provide  organization  or  general  structure  for  the  subject 
matter.  Duchastel  and  Merrill  also  pointed  out  that  objectives  may 
serve  a  management  function  by  enabling  the  students  to  organize  their 
time  and  learning  experiences  better  In  terms  of  the  goals  of  the  course. 
This  might  help  eliminate  the  typical  cramming  sessions  which  often  precede 
tests,  though  hard  evidence  is  as  yet  lacking.  Another  function  is  that 


of  providing  learners  feedback  i '*  terns  of  the  criteria  set  forth  in  the 
objective,  enabling  students  to  deal  with  any  discrepancies  between 
performance  and  goal.  Finally,  presenting  objectives  to  students  may  help 
to  motivate  them.  Stjdents  who  know  that  they  have  satisfactorily  met 
the  criteria  set  forth  in  the  objective  will  probably  be  more  motivated 
than  students  whose  only  reinforcement  comes  from  a  grade  at  the  end 
of  a  course.  Duchastel  and' Merrill  also  pointed  out  that  presenting 
objectives  to  students  will  have  no  results  If  the  students  pay  no 
attention  to  them  In  the  learning  situation.  Therefore,  a  teacher  must 
make  an  effort  to  explain  thoroughly  the  meaning  of  objectives  to 
students  so  that  they  will  actually  use  them  while  learning.  However, 
the  teacher  must  also  avoid  giving  students  such  long  and  extensive  lists 
that  they  are  overwhelmed  or  confused. 

Gagne  and  Briggs  (1979}  ag^ee  that  the  advantage  of  providing 
objectives  to  students  Is  that  It  Informs  the  learners  of  their  goal. 

They  disagree  with  those  who  contend  that  when  an  Instructor  gives  an 
objective  to  students,  they  may  be  inhibited  from  trying  to  meet  still 
other  worthwhile  objectives  which  they  may  formulate  themselves. 

Pros  and  Cons 

The  arguments  for  and  against  the  use  of  performance  objectives  arc 
usually  advanced  In  terms  of  three  criteria: 

To  what  extent  do  they  help  a  student?  (Mh*yi  students  are  given 
performance  objectives,  do  they  perform  any  better?) 

To  what  extent  do  they  help  a  teacher?  (When  Instructors  are  given 
performance  objectives,  or  trained  to  generate  performance 
objectives,  do  they  instruct  any  better?  or  do  their  students 


Is 


learn  no re  and/or  faster?) 

Tc  wnat  extent  do  they  help  an  instructions  designer?  (Wneo  ISC' 
personnel  are  given  performance  objectives,  or  trained  to 
generate  their  own,  do  they  design  better  Instruction?) 

As  we  have  mentioned  earlle*',  the  published  research  is  both  meager  and 
mediocre.  Nevertheless,  even  though  we  cannot  definitive  answers 
to  these  three  questions  in  the  literature,  we  can  identity  (albeit 
tentatively)  apparent  trends;  even  more  important,  the  literature  clearly 
leads  us  to  several  researc!  endeavors. 

Students.  Several  studies  have  yielded  data  that  led  their  authors 
to  conclude  that  students  who  have  performance  objectives  prior  to 
Instruction  achieve  more,  in  terms  of  posttest  performance,  than  qv  other 
students  (Bassett  and  Klbler,  1975;  Dalis,  1970;  Doty,  1968;  Lawrence, 

1970;  Engel,  1968;  Blaney  and  McKle,  1969;  Tiemar.,  1968).  Aside  from 
other  deficiencies,  not  one  of  these  studies  included  an  operational 
definition  of  the  Independent  variable.  It  hardly  seems  necessary  to 
call  attention  to  the  irony:  experimenters  conducting  research  on  the 
value  of  communicating  to  learners  In  clear  and  unequivocal  terms  the 
purpose  of  the  Instruction  themselves  fail  to  provide  a  clear  and 
unequivocal  definition  of  the  phenomenon  under  investigation.  The  fact 
that  a  considerable  nwnber  of  studies  Indicates  that  availability  of 
performance  objectives  to  students  makes  no  significant  difference 
(Boardnan,  1970;  Smith,  1967;  Weinberg,  1970-  to  mention  only  three) 
and  that  another  considerable  niinber  indicates  that  students  who  we  0 
provided  performance  objectives  prior  to  instruction  performed 
on  a  posttest  than  did  other  students  (Olson,  1971;  felo'1  -  Schmidt,  1971, 
Stedman,  1970)  adds  nothing  to  our  knowledge;  the.£?  studies,  too,  are  plagued 
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b'/  the  same  deficiency  —  lack  of  a  definlt  on  of  a  performance  objective. 

Di icbastel  and  Merrill  (1973)  and  MacDonald-  oss  (1973)  ntimated,  if  they 
die  not  so  state,  that  one  could  not  use  th>  Informatio  included  In  most 
put >1 i shed  articles  to  determine  whether  a  9  ven  statemert  is  or  is  not  a 
performance  objective.  Seme  of  these  sludi.'S  used  Mager  s  three  criteria; 
this  is  no  help,  for  one  can  adhere  strict!/  to  these  criteria  and  still 
product  a  set  of  objectives  which,  at  best,  would  be  classified  as  examples 
of  a  gr.oss  and  unanalyxed  variable. 

Another  problem  that  most  studies  cf  this  type  have  failed  to  recognize 
is  that  level  of  specificity  may  be  crltl cal .  MacDonald- toss  has  pointed 
out  that  there  are  no  rules  for  determining  degree  of  level  of  specificity. 
Clearly  some  effort  should  be  redirected  toward  formulating  such  a  set  of 
rules,  lest  we  discover  that  availability  of  performance  objectives 
facilitates  learning,  and  then  have  no  use  for  fhe  findings  because  we  do 
not  Know  how  general  or  how  specific  to  make  the  performance  objectives. 

There  seems  little  point  In  analyzing  the  studies  11  terns  of  design 
and  statistics;  good  design  and  statistics  are  worthless  If  the  independent 
varicole  cannot  be  reproduced  or  replicated. 

etchers.  There  appears  to  be  an  even  greater  dearth  of  empirical 
ev^denw  to  support  the  hypothesis  that  performance  objectives  help  teachers 
teach  bitter,  regardless  of  whether  the  dependent  variable  is  teacher 
behavior  or  student  performance.  It  hardly  requires  a  research  study  to 
come  to  tie  conclusion  that  even  If  instructors  are  given  performance 
objective;  (or  taught  to  construct  good  performance  objectives),  their 
students  will  not  achieve  a  satisfactory  criterion  when  subjected  to  bad. 
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or  even  mediocre.  Instruction.  Sullivan  and  Niedermeyer  (1977)  pointed 
out  (not  in  a  research  report!)  that  if  teachers  want  students  to  achieve 
mastery  of  objectives,  teachers  must  also  be  provided  with  instructional 
materials  and  procedures  developed  especially  for  the  objectives  of  the 
lesson.  Merely  providirj  performance  objectives  to  the  teachers  may  have 
little  effect  on  either  the  instruction  or  the  learning.  Teachers  should 
not  be  held  aceountablefor  high  levels  of  pupil  performance  on  objectives 
without  the  proper  development  of  Instructional  materials  and  procedures 
based  on  the  objectives.  They  concluded  that  with  the  Increasing  number 
of  objectives -based  programs  In  our  schools,  there  Is  need  for  empirical 
evidence  to  support  the  concept  of  Objective-Based  Instructional  Programs. 
We  might  add  that  there  Is  also  need  for  empirical  evidence  to  support 
the  widespread  claims  for  the  benefits  of  jslng  performance  objectives, 
period  —  but  it  could  be  inferred  that  the  authors  meant  to  say  tills  too. 

Several  studies  of  the  effect  of  providing  Instructors  with 
performance  objectives  are  worth  examining.  Baker  (1969)  gave  teachers 
objectives  in  an  Investigation  of  the  effects  of  behavioral  and  non- 
behavloral  objectives  on  students'  achievement.  She  randomly  asslwid 
one  of  three  lists  of  objectives—  one  list  non-behavloral ,  the  otwr  twc 
lists  behavioral—  to  high  school  social  studies  teachers  and  asked  then 
to  teach  for  the  attainment  of  the  objectives  in  their  classrooms  No 
significant  differences  were  found  between  posttest  scores  of  students 
In  the  behavioral  and  non-behavloral  treatment  groups.  Baker  suggested 
that  the  teachers'  faulty  understanding  of  objectives,  Indicated  by  their 
observed  Inability  to  provide  relevant  classroom  practice  and  to  Identify 
appropriate  test  items,  may  have  accounted  for  the  lack  of  difference 


among  the  groups.  Baker  suggested  that  a  replication  of  the  study  be 

conducted  ‘-rfth  one  group  of  teachers  thoroughly  trained  In  the  use  of 

& 

behavioral  objectives  and  another  group  of  teachers  not  so  trained. 

Jenkins  and  Dono  (1971)  investigated  whether  providing  teachers, 

students,  or  both  teachers  and  students  with  general  or  specific  performance 

objectives  increased  the  amount  learned.  The  results  did  not  support  the 

hypothesis  that  type  (general  or  specific)  of  objective  affected  learning. 

Furthermore,  neither  condition  was  significantly  superior  to  conditions 

In  which  performance  objectives  were  not  provided.  They  concluded: 

The  possibility  remains,  however,  that  type  and  knowledge  of 
oojectives  were  insignificant  variables  because  they  received 
Inadequate  attention  from  both  the  t.e?chers  and  the  students. 

Since  teachers  and  students  rarely  are  exposed  to  the  explicit 
objectives  of  Instruction  they  might  fall  to  use  these  objectives 
ipproprlately  either  because  their  value  Is  not  recognized  or 
oecause  one  must  learn  how  to  use  explicit  objectives. 

Moreover,  Jenkins  and  Deno  suggested  that  more  benefit  may  be  derived 

If  teachers  and  students  were  given  some  Incentive  to  use  the  oojectlves 

or  were  given  practice  In  their  use. 

The  effects  of  "pre-pared"  and  teacher-prepared  performance 

objectives  on  the  learning  of  educablt  mentally  retarded  children  were 

investigated  by  Crooks  (1971).  He  used  three  groups: 

(1)  Teachers  were  given  and  used  "pre-pared"  performance  objectives; 

(2)  Teachers  prepared  and  used  their  own  objectives,  lesson  plans, 
and  activities; 

(3)  Teachers  used  their  own  Instructional  approach. 

Significant  differences,  In  terms  of  pupils'  posttest  scores,  were  found 
between  the  groups  who  used  "pre-pared"  objectives  and  those  vrfio  prepared 
their  own  objectives.  Students  scored  higher  In  the  groups  who  were 
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taught  by  teachers  who  used  Hpre~pared“  objectives.  The  results  indicated 
an  advantage  1r»  using  "pre-pared"  objectives  and  materials  Unfortunately* 
no  mention  was  made  of  whether  or  not  the  teachers  were  trained  to  use 
the  objectives  provided. 

The  next  category  of  studies  included  the  training  of  teachers  In 
the  use  of  performance  objectives  and  the  effect  of  such  training  on 
students’  achievement.  The  type  of  training  and  the  amount  of  time  devoted 
to  training  varied  fron  study  to  study.  In  some  cases,  It  was  difficult 
to  ascertain  the  precise  nature  cf  the  training.  Two  studies  In  this 
group  showed  that  training  In  the  use  of  performance  objectives 
facilitated  students'  learning,  while  two  did  not.  Although  some  studies 
failed  to  find  that  the  training  of  teachers  In  the  use  of  objectives 
Increased  student  learning,  there  were  —  according  to  the  researchers  — 
some  indications  that  favored  this  hypothesis. 

Platt  (1969)  Investigated  the  effect  of  training  teachers  In  defining, 
writing,  and  implementing  performance  objectives  on  learner  outcomes  In 
a  seventh  grade  mathematics  program.  In  this  study  he  measured  mathonatlcs 
skills*  mathematics  application,  and  student  attitudes.  He  used  the 
Stanford  Achievement  lest  to  measure  mathematics  skills  and  application, 
and  the  Hayes  Pupil -Teacher  Reaction  Scale  to  measure  the  students ' 
attitudes  toward  their  teachers.  In  the  areas  of  mathematics  skills  and 
students*  attitudes,  Piatt  found  that  pcsttest  scores  of  students  In  the 
group  that  received  the  specific  training  were  significantly  higher 
than  those  of  students  In  the  other  group.  However,  he  did  not  find  this 
effect  In  the  area  of  mathematics  application.  Thus,  Piatt  was  able  to 
demonstrate  that  the  training  in  the  use  of  performance  objectives 
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provided  &  facultative  effect.  In  terms  of  student  posttest  performance, 
but  not  in  all  subject-matter  areas. 

Sryant  (1970)  Investigated  the  effects  of  performance  objectives  on 
the  achievement  level  of  low  achieving  eighth  grade  science  pupils  in 
predominantly  black  Inner-city  schools.  He  found  that  pupils  taught 
by  teachers  trained  in  the  use  and  development  of  performance  objectives 
achieved  higher  scores  on  the  criterion  measure.  In  addition,  the  use 
of  performance  objectives  appeared  to  facilitate  the  presentation  of 
course  content  in  small  manageable  pieces  and  It  provided  more  opportunity 
for  the  lew  achiever  to  experience  success  In  the  classroom.  However, 
the  criterion  measure  used  In  this  study  was  spurious:  It  was  developed 
by  both  the  trained  and  the  untrained  teachers  and  Its  validity  as  a 
measuring  device  was  questionable. 

Cardarelll  (1971)  provided  teachers  with  training  In  recognizing  and 
writing  objectives.  Identifying  appropriate  activities,  and  constructing 
test  Items  matched  to  the  objective.  Each  teacher  received  the  same 
materials  on  the  topic.  Teachers  could  use  all,  some,  or  none  of  the 
resources  given  them.  Training  did  not  have  a  significant  effect  on 
students'  achievement.  Pupils  of  those  teachers  who  received  training 
but  were  assigned  non -behavioral  objectives  achieved  the  highest  mean 
score  on  the  criterion  measure.  This  result  was  explained  by  the  presumption 
that  teachers  have  a  greater  commitment  to  achievement  of  a  goal  when  they 
write  their  own  objectives.  On  the  other  hand,  pupils  of  teachers  who 
received  behavioral  objectives  but  no  training  had  the  lowest  mean  score 
on  the  criterion  measure.  It  was  suggested  that  without  training,  teachers 
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do  not  perceive  behavioral  objectives  and  non-behavio  al  objectives 
differently;  consequently,  teichers  should  be  trained  but  allowed  to 
write  their  own  objectives  rather  than  being  assigne*.  objectives.  The 
results  of  this  study  contradict  those  of  an  earlier  study  j Crooks, 

1971)  In  which  the  use  of  prepared  objectives  and  materials  was  found 
to  be  facllitatlve. 

Cllngman  (1972)  investigated  the  effect  cf  providing  teachers  and 

students  with  performance  objectives  on  the  amount  of  learning,  frequency 

* 

of  course  attendance,  and  student  satisfaction  All  teacl»ers  were 
given  traditional  content  outlir.es  for  oach  topic.  The  treatment  group, 
both  teachers  and  students,  received  statements  of  educational  objectives. 
The  teachers  attended  a  one-day  acinar  directed  by  the  Investigator,  who 

also  met  once  with  the  students  to  answer  questions  about  the  statements 

* 

of  objectives.  The  differences  were  not  significant.  Cllngnan  suggested: 

(1)  Students'  and  teachers'  un familiarity  with  performance 
objectives  and  level s-of-leamlng  concept  made  the 
communication  of  objectives  difficult  at  other  than 
the  recognition-recall  level; 

(2)  the  ability  to  understand  and  utilize  objectives  appeared 
to  be  acquired  slowly; 

(3)  most  of  the  students  favored  the  Idea  of  receiving 
objectives; 

(4)  most  of  the  students  In  the  treatment  group  reported 
that  they  were  unable  to  us*  the  objectives  In  a 
meaningful  way. 

Since  the  emphasis  was  not  placed  on  the  training  of  the  teachers,  it 
was  difficult  to  ascertain  the  Influence  of  training  on  the  results 
reported  in  this  study.  Perhaps  the  training,  per  se,  was  Inadequate. 
Additional  findings  reported  by  the  Investigator  suggested  a  need  for 
more  specific  training  in  the  use  of  behavioral  objectives  for  both 
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teachers  and  students. 

In  stannary:  the  literature  cited  falls  into  one  of  two  categories 
of  research  studies:  (1)  the  Investigations  of  the  effects  on  learning 
when  teachers  possessed  the  objectives;  (2)  the  investigations  of  the 
facultative  effect  that  the  training  of  teachers  in  the  use  of 
performance  objectives  had  on  students'  achievement.  The  first  group 
of  studies  (N  *  6)  did  not  show  significant  effects  on  students' 
achievement  as  a  function  of  teachers'  possession  of  objectives.  In 
the  second  group  of  studies  (N  ■  4)  two  Investigators  (Platt,  1969; 

Bryant,  1971)  found  that  the  training  of  teachers  In  the  use  of  objectives 
had  a  facultative  effect  on  students'  achievement.  This  facultative 
effect  was  not  found  In  the  other  two  reported  studies  (Gardarelll,  1971; 
Cllngman,  1972).  Consequently,  substantial  support  was  not  provided 
for  the  training  of  teachers  in  the  use  of  behavioral  objectives. 

Although  these  ten  studies  used.  In  the  main,  better  designs  and 
data  analyses  than  did  the  studies  cited  In  the  section  preceding  this 
one,  ill  failed  to  define  the  independent  variable  performance  objective 
adequately.  Indeed,  It  would  be  Impossible  to  replicate  any  one  of  them, 
given  the  definitions  of  performance  objective  published  In  these  studies. 

Form  of  Performance  Objectives 

Nearly  all  writers  have  asserted  that  there  are  three  components  of 
a  performance  objective:  a  verb  which  describes  the  learner  performance 
or  activity;  a  statement  of  the  condltlon(s)  under  which  this  performance 
takes  place;  and  a  statement  of  toe  standard  by  which  toe  performance 
Is  judged  or  evaluated.  All  writers  agree  that  these  three  components 
must  be  stated  In  clear,  unequivocal  terras  (Tyler,  1934;  Mager,  1962; 
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Llndval ,  1964;  Bloon,  1564;  Klbler,  Barker,  and  Miles,  1970;  Popham 
and  Baker,  1970;  GerUch  and  Ely,  1971).  Mager's  criteria  have  become 
so  widely  accepted  that  they  are  worth  reproducing  here; 

(1)  One  should  state  the  objective  In  terms  of  what  the  learner 
will  be  able  to  do  after  the  learning  experience.  This  Is  done  by 
selecting  vens  which  describe  observable  actions.  Such  words  as 
Identify,  describe,  construct,  aid  list  are  far  less  ambiguous 
than  verbs  Mich  as  to  know,  understand ,  or  appreciate. 

(2}  The  s:Cond  characteristic  of  a  well-stated  objective  Is  a 
statement  of  the  conditions' under  wi  ch  the  performance  is  to 
occur.  .‘indltlons  should  be  stated  c 'early  enough  that  others 
understand  your  Intent  as  you  understand  It. 

(3)  Tne  third  characteristic  of  a  well-$tav.ed  objective  is  the 
crltrrlon,  the  quality  or  level  of  perfomtanre  that  will  be 
considered  acceptable. 

In  Ch'j  years  that  have  passed  snce  their  original  publication  In 
19<2,  these  criteria  have  been  subjected  to  minor  criticism.  Merrill 
(19  <)»  fcr  example,  has  charged  tha;  thase  criteria  fall  to  provide 
fr*  distinguishing  between  or  among  levels  of  behavltr,  Another 
ejection  raised  by  Merrill  Is  that  there  are  almost  always  two  aspects 
of  conditions  under  which  behavior  o:curs.  The  first  Is  concerned  with 

those  conditions  related  to  a  parti cilar  subject  nutter  and  unique  to 

« 

the  testing  situation.  An  example  c*  a  condition  stated  In  a  beha.ioral 
objective  for  a  mathematics  class  wold  be  "...  ustnj  only  a  calculator.. N 
or  "...  using  only  the  protractor.. The  second  are  the  psychological 
conditions  which  help  define  the  behvlor  being  observed.  In  most  cases 

\ 
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the  psychological  conditions  are  not  stated  In  the  objective,  but  have 
an  important  effect  upon  its  mean. ng,  This  second  type,  quite  often 
overlooked,  is  the  more  important  of  the  two,  since  the  type  of  behavior 
being  observed  will  change  when  psychological  conditions  are  changed, 
for  t<ample,  the  classroom  learning  environment  is  typical!  v  not  normal 
the  da^  before  Christmas  vacation  begins. 

The  verb.  The  requirements  for  verbs  have  been  expanded  considerably 
by  Gagne  and  Briggs  (1979) ,  wPo  stated  that  verbs  must  be  present  in 

every  good  performance  objective.  The  first  verb  denotes  action:  writes, 
draws,  selects,  matches,  names,  groups,  verifies;  there  are  many  others. 

The  following  example  denotes  acti on:  'Without  use  of  reference  materials, 
state  the  provisions  of  the  Fifti  Amendment,  in  writing."  While  the 
action  verb  may  be  essential  for  effective  communication,  it  is  not 
necessarily  the  most  important  verb  in  an  objective.  The  second  verb 
(the  ^ajcr  verb)  denotes  learned  capability.  It  describes  the  to-be-learned 
human  capability  as  it  may  be  observed  in  some  performance  exhibited  by 
the  learner;  discriminate,  classify,  demonstrate,  generate,  execute, 
originate,  identify,  and  state  are  examples.  A  Gagne-8riggs  performance 
objective,  then,  looks  like  these: 

"identifies,  by  naming,  the  subject,  direct  object,  and  indirect 
object.. ." 

"demonstrates,  by  solving  orally  presented  examples,  t*e  addition 
of..." 

"generates,  by  synthesizing  applicable  rules,  a  paragraph. 

The  emphasis  on  the  observability  of  the  verb  has  led,  quite  naturally, 
to  lasts  of  recommended  verbs.  Some  of  the  most  highly  recommended  verbs 
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are  "Identify,  name,  describe,  construct,  state,  discriminate,  classify, 
generate,  name,  order,  check,  and  perform."  (Commission  on  Science 
Education,  1965;  Gerlach,  Sullivan,  Baker,  and  Schutz,  1966;  Briggs,  1970). 
Either  explicitly  or  implicitly,  such  writers  warn  against  the  use  of 
"vague"  verbs  such  as  know,  understand,  appreciate. 

While  the  performance  standard,  either  Implied  or  expressed,  Is 
frequently  100*  (rdnus  some  undesignated  "measurement  error"),  it  Is  not 
unusual  to  find  a  lower  standard;  for  example,  "80*  of  the  class  will 
achieve  at  least  BO*  accuracy,"  or  "three  out  of  five  problems  solved 
correctly,"  or  " four  out  of  six  defects  Identified"  (Bloom,  19/1).  Briggs 
(1970)  stated  that  when  the  evaluation  of  the  learner's  performance  Is 
complex,  It  may  be  preferable  to  omit  the  standard  in  the  performance 
objective  and  to  present  It  in  the  test  scoring  key  or  In  Ue  grade 
conversion  guide.  Another  quite  different  approach  to  stating  the 
standard  of  perfomance  is  the  mastery  approach  described  by  Gagn£  and 
Briggs  (1S79).  They  asserted  that  there  are  two  compelling  reasons  for 
omitting  the  standard:  (a)  the  standard  Is  not  likely  to  be  applied  in 
the  same  manrer  to  all  Individuals,  and  (b)  the  question  of  performance 
standard  is  a  question  of  measurement  and  is  integral  to  the  problem 
of  performance  assessment.  They  contend  that  a  concern  for  assessment 
procedure  at  the  time  when  performance  objectives  are  being  generated 
will  cause  confusion.  Rather,  they  assert,  sines  learning  is 
hierarchical  \t  nature,  and  since  the  acquisition  cf  a  higher  order 
behavior  Is  continent  upon  the  mastery  of  all  relevant  lower  order 
behaviors.  It  Is  unwise  to  adopt  an  arbitrary  standard  such  as  "five 
sue  of  six  correct  rvsponses."  Mastery  criteria  vary  from  one  objective 


to  another;  they  need  to  be  determined  as  part  of  the  assessment  process. 

Despite  these  differences,  the  form  of  the  performance  objective 
recommended  by  Mager  in  1962  has  almost  become  dogma  in  education  and 
training.  However,  there  are  also  those  who  have  “left  the  church." 
Typical  of  their  heretical  statements  Is  the  assertion  that  there  Is  no 
point  In  requiring  that  a  performance  objective  be  written  in  a  standard 
form  unless  the  purpose  for  writing  the  objective  is  considered  first. 
There  will  be  a  wholesome  variation  In  form  between  teacher-constructed 
objectives;  the  Mager  model  may  help  some  writers  while  it  may  be 
totally  unnecessary  for  others  (Harlen,  1972). 

Summary.  The  review  of  the  literature  leads  to  the  following  three 
conclusions: 

(1)  There  Is  a  general. agreement,  not  supported  by  research,  that 
performance  objectives  are  useful  for  learners.  Instructors,  and 
designers. 

(2)  Although  there  Is  a  solid  consensus  regarding  the  definition 
•of  Performance  objective,  the  definition  is  not  operational. 

(3)  While  there  Is  general  agreement  concerning  the  most  desirable 
form  of  a  performance  objective,  examples  of  objectives  which 
possess  the  three  formal  characteristics  may  differ  greatly  from 
one  another. 

As  a  result  of  these  findings,  we  decided  to  begin  experimenting  In 
an  effort  to  develop  a  behavioral  (or  operational)  definition  of  the  term 
performance  objective. 
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Study  1 

Designer's  manuals  as  well  as  research  reports  have  focused  almost 
entirely  on  the  role  of  the  verb  In  a  performance  objective.  The 
statements  of  condition  and  criterion  have  been  more  or  less  taken  for 
granted.  This  emphasis  seems  to  have  been  based  on  the  assumption  that 
the  verb  Is  the  primary,  If  not  the  sole,  determiner  of  the  degree  to 
which  an  objective  may  be  considered  behavioral,  since  It  Is  the  verb 
that  designates  what  the  .learner  does.  Lists  of  "observable"  verbs 
began  to  appear.  The  Implicit,  If  not  explicit,  guarantee  was  that 
when  such  verbs  are  used  to  construct  objectives,  the  objectives  will 
be  "good,"  "worthwhile,"  "precise,"  "observable,"  and  the  like. 
Conversely,  use  of  a  verb  not  on  a  recommended  list  would  automatically 
render  an  objective  suspect  (1.e.»  non-behavloral ,  to  be  avoided). 

Deno  and  Jenkins  (1969)  appear  to  have  been  the  first  to  provide 
evidence  that  tha  behavioral  objectives  cited  In  developmental  curricula 
are  not  necessarily  statements  of  highly  observable  behaviors.  They 
attribute  this  Inconsistency  to  the  presumption  that  curriculum 
developers,  both  to  avoid  excessively  lengthy  documentation  ana  to  avoid 
the  appearance  of  triviality,  have  moved  to  wore  general  terms  for  which 
the  referents  are  less  directly  observable.  They  asked  14  In-service 
educators  to  rate  the  observability  of  99  verbs  drawn  from  the  objectives 
of  a  "widely  cited"  experimental  curriculum  on  •  scale  from  1  (highly 
observable)  to  S  (non-observable).  (See  Appendix  A  for  Instrument.) 

The  results.  In  Table  1-1,  show  clearly  that  many  coHmonly  used  verbs 
received  Intermediate  ratings  (around  3  on  a  1  to  5  scale),  and  that 
almost  half  of  the  "action*  verbs  received  ratings  toward  the 
nonobservable  end  of  the  scale  (3  to  5).  In  Itself,  this  finding 
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would  hardly  be  worthy  of  mention.  What  is  noteworthy  is  that  many  of 
the  verbs  that  the  advocates  of  behavioral  objectives  assert  are  open 
to  few  misinterpretations  (e.g.,  deduce,  determine,  create,  solve,  round 
off*  multiply)  had  mean  ratings  greater  than  3.0  and  the  ratings  of  many 
of  the  verbs  revealed  surprisingly  large  variances  {e.g.,  state,  Identify 
in  writing,  and  round  o*f  each  had  variances  of  1.4). 

The  conclusion  drawn  by  Oeno  and  Jenkins  was  that  verbs  used  in 
behavioral  objectives  an  chosen  for  consistency  of  usage  rather  than 
maxlmun  observability  per  se,  and  that  the  terms  "behavioral M  and 
"observable,"  though  nlated,  an  not  synonyms.  It  Is  nasonable  to 
suggest  that  other  characteristics  of  words,  such  as  clarity,  precision, 
and  concreteness  an  also  at  work  In  detarminlng  the  choices  of  curriculum 
developers. 

Because  of  the  unexpected  and  striking  natun  of  the  Oeno  and 
Jenkins  findings,  we  felt  seme  pressun  to  verify  their  nsults;  accordingly, 
we  npllcated  their  study.  We  wen  concerned  with  whether  pre-service 
and  in-service  teachers  In  an  education  ;*jursa  would  rate  the  verts  In 
a  similar  fashion.  Subjects  wen  35  senior  and  first-year  graduate 
students  in  an  audiovisual  education  course  at  the  University  of  Minnesota. 
We  npnduced  the  original  instrument,  without  change,  and  administered 
it  during  the  fburth  weekly  meeting  of  the  course. 

the  results,  shewn  In  Table  1-1,  an  essentially  the.  same  as  those 
obtained  by  Oeno  and  Jenkins.  The  correlation  between  th?  two  sets  of 
mean  ratings  was  r  •  .89,  while  the  correlation  between  tin  two  sets  of 
ratings,  by  rink,  was  r'  «  ,91. 

Obviously,  the  taphasls  that  advocates  of  behavioral  objectives 


have  placed  on  "observable"  or  "unambiguous"  verbs  is  open  to  question. 


Study  2 


Problem 

The  results  of  Study  1  led  us  to  wonder  whether  or  not  the  ratings 
would  persist  If  we  embedded  the  verbs  In  statements  of  objectives. 
Furthermore,  since  objectives  generally  consist  of  a  statement  of 
condition  and  a  statement  of  criterion  In  addition  to  a  verb,  we  wished 
to  learn  what  role  these  two  components  would  play  In  determining  raters' 
perceptions  of  objectives.  Consequently,  we  decided  to  seek  answers  to 
these  two  quest lens: 

(1)  Can  we  instruct  a  set  of  objectives  which  will  be  rated  high 
(l.e.,  1.0  or  close  to  1.0)  to  low  (l.e.,  5.0  or  close  to  5.0) 
on  tfw  "most  observable— least  observable"  scale  and  which 
will  have  low  variances? 

(2)  If  sc,  can  we  ascertain  what  characteristics  account  for  the 
ratings  which  the  objectives  receive? 


Procedures 

Twenty-one  senior  and  first-year  graduate  students  enrolled  In  an 
audiovisual  education  course  at  the  University  of  Minnesota  completed  a 
four-part  questionnaire.  Part  I  consisted  of  the  99  verbs  rated  In  the 
Oeno-Jenklns  study  (1969).  Part  II  consisted  of  17  expressions  usable 
as  statements  of  conditions  In  objectives  (e.g.,  "given  previously 
unencountered  examples").  Part  III  consisted  of  19  expressions  usable 
as  statements  of  criterion  (e.g.,  "with  90*  accuracy").  Part  IV  consisted 


23 


of  53  expressions  disced  into  four  types: 

Type  1:  Six  "verb-only"  statements. 

Type  2'  12  statements  containing  a  verb  and  a  statement  of 

condi ti or s. 

Type  3::  12  statements  containing  a  verb  and  a  statement  of 
criterion. 

Type  4:  23  complete  behavioral  objectives  (verb  plus  condition 

plus  criterion). 

The  items  in  Parts  II  and  III  were  generated  by  the  investigators , 
who  subjectively  selected  expressions  which  they  considered  either 
"precise"  or  "vague,"  •  The  items  in  Part  IV,  Types  2,  3,  and  4,  were 
generated  by  combining  a  verb  from  Part  I  wit.n  a  condition  statement  from 
Part  II  and/or  a  criterion  statement  from  Part  III.  The  subjects  were 
instructed  to  rate  each  item  in  Parts  I  and  IV  on  a  "most  observable- 
least  observable"  scale  from  1  to  5,  respectively.  The  items  in  Parts  II 
and  II!  were  to  be  rated  cn  a  similar  five-point  scale  from  "most  precise" 
(1 )  to  "most  vague"  (5). 

Means  and  variances  were  computed  for  each  item  on  the  questionnaire. 
Correlations  were  then  computed  between  and  among  the  components  and  the 
total  statements. 

Results 

Table  2-1  shows  the  means  and  variances  for  each  item  in  Parts  I,  II, 
III,  and  IV  respectively. 

A  multiple  correlation  of  .83  was  obtained  between  the  mean  ratings 
of  the  complete  statements  and  the  three  components  of  each  statement. 
Intercorrelations  of  mean  ratings  for  the  complete  statements  and  each 
of  their  components  are  given  in  Table  2-2.  These  correlations  are  based 


Table  2-1 

Means  and  Variances  of  Ratings  Obtained  ’"n  Study  2 


Part  I  Verb  Mean  Variance 


1. 

Tc  cover  with  a  card 

1.0 

0.0 

2. 

Tc  point  to 

1.1 

0.1 

3, 

To  line  draw 

1.1 

0.2 

4. 

To  raark 

1.2 

0.5 

5. 

To  lever  press 

1.3 

0.4 

6. 

To  underline 

1.0 

0.0 

7. 

To  cross  out 

1.1 

0.1 

8. 

To  walk 

1.1 

0.1 

9. 

To  circle 

1.0 

0.0 

10. 

Tc  repeat  orally 

1.2 

0.3 

11. 

To  count  orally 

1.1 

0.3 

12. 

To  say 

1.4 

0.4 

13. 

To  write 

1.0 

0.0 

14. 

To  put  on 

1.5 

0.6 

15. 

To  read  orally 

1.0 

0.0 

16. 

To  shade 

1.6 

0.6 

17. 

To  number 

1.1 

0.1 

18. 

To  name 

1.4 

0.7 

19. 

To  fill  in 

1.4 

C.5 

20. 

To  label 

1.3 

0.4 

21. 

To  state 

1.4 

1.0 

22, 

To  move 

1.3 

0.4 

20. 

Tc  place 

1.5 

0.9 

24. 

To  tell  what 

1.6 

0.6 

2i. 

To  draw 

1.1 

(.3 

21. 

To  identify  in  writing 

1.3 

0.5 

2/. 

To  check 

1.9 

1.2 

28. 

To  construct 

1.3 

0.4 

29. 

To  et&tch 

1.6 

1.1 

30. 

To  taka  away 

1.4 

0.5 

31. 

To  sake 

1.3 

0.3 

32. 

To  arrange 

1.7 

0.8 

33. 

To  finish 

2.1 

1.5 

34. 

To  read 

2.2 

1.9 

35. 

To  play 

2.0 

1.2 

36. 

To  locate 

1.5 

0.7 

37. 

To  connect 

1.2 

0.5 

38. 

To  give 

1.2 

0.5 

39. 

To  reject 

2,0 

1.1 

40. 

To  select 

1.7 

1.1 

41. 

To  choose 

1.7 

0.9 

42. 

To  partition 

1.6 

1.3 

43. 

To  change 

2.6 

1.8 

44. 

To  use 

2.4 

1.4 

45. 

To  subtract 

‘  1.5 

0.9 

46. 

To  perform 

1.5 

0.8 

47. 

To  total 

1.4 

0.5 

48. 

To  divide 

1.5 

0.7 

Table  2-1  (continued) 


Verb 

Mean 

Variance 

49.  To  order 

1.6 

1.2 

50.  To  measure 

1.4 

0.4 

51.  To  add 

1.6 

1.1 

52.  To  supply 

1.9 

0.6 

53.  To  demonstrate 

1.9 

0.9 

54.  To  regroup 

2.0 

1.1 

55.  To  multiply 

X.5 

0.9 

56.  To  round  off 

1.8 

1.0 

57.  To  group 

1.4 

0.6 

58.  To  complete 

1.9 

1.3 

59.  To  respond  to 

2.3 

1.8 

60.  To  average 

1.9 

1.5 

61.  To  summarize 

2.5 

1.7 

62.  To  Inquire 

2.6 

1.9 

63 .  To  utilise 

2.9 

2.0 

64.  To  borrow 

1.9 

0.9 

65.  To  acknowledge 

3.1 

1.5 

66.  To  find 

1.9 

1.3 

67.  To  identify 

1.5 

0.7 

68.  To  sea 

2.7 

2.3 

69.  To  convert 

2.3 

1.7 

70.  To  distinguish 

2.7 

1.9 

71.  To  solve 

2.5 

1.9 

72.  To  apply 

3.4 

1 .4 

73.  To  develop 

3.6 

1.2 

74.  To  test 

2.5 

1.9 

75.  To  determine 

3.7 

1.5 

76.  To  generate 

3.6 

1.7 

77 .  To  create 

?..6 

2.0 

78.  To  discriminate 

2.9 

1.9 

79.  To  recognize 

3.1 

1.7 

80.  To  discover 

3.2 

1.5 

81.  To  became  competent 

3.9 

1.4 

82.  To  infer 

4.2 

C.9 

83.  To  like 

4.2 

1.4 

84.  To  analyze  ' 

3.8 

1.4 

85.  To  be  curious 

4.3 

1.0 

86.  To  conclude 

3.3 

2.2 

87.  To  deduce 

3.5 

1.8 

88.  To  feel 

4.3 

1.5 

89.  To  concentrate 

4.2 

1.3 

90.  To  perceive 

4.4 

0.9 

91.  To  Stink 

4.6 

0.9 

92.  To  think  critically 

4.4 

1.3 

93 .  To  learn 

4.1 

1.3 

94.  To  appreciate 

4.6 

C.5 

95.  To  be  aware 

4.3 

1.2 

96.  To  know 

4.6 

0.9 

97.  To  wonder 

4.7 

0.4 

98.  To  realize  fully 

4.6 

0.7 

99.  To  understand 

4.5 

0.9 

Table  2-1  (continued) 


Part  IV 

1. 

2. 

* 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

JO. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 
21. 
22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 


Objective  Statement  Mean 


Tc  cover  x  &  y  with  a  card  1-0 

To  remove  x  &  y  with  no  more  than  one  error  given 

pliers,  screwdriver  and  hammer  1*1 

To  write  x  &  y  from  memory  1.1 

To  lever  press  either  x  or  y  within  two  seconds  1-1 

To  point  to  x  &  7  with  1002  accuracy  1.1 

To  count  orally  to  x  &  y  1*2 

To  play  x  &  y  with  1002  accuracy  from  memory  1.2 

To  determine  x  &  y  with  no  more  than  one  error, 

given  a  tape  measure,  a  3*  chain  and  a  compass  1-3 

To  supply  x  &  y,  given  specific  tools  or  equipment  1-4 
To  read  x  6  y  without  error  1*5 

To  convert  x  &  y  on  the  first  attempt,  given  a 

correctly  solved  example  for  reference  1.5 

To  draw  x  &  y  without  any  reference  to  help  1-6 

To  round  off  x  &  t  on  the  first  a:tempt  1-8 

To  demonstrate  x  &  y,  given  exemplars  and  noa- 

exmplars  of  correct  demonstrations  1.8 

To  line  draw  x  6  y  consistently,  given  previously 

unencountered  tasks  1-8 

To  label  x  6  y  without  error,  given  visible 

encouragement  1*9 

To  check  x  6  y  2.0 

To  circle  x  6  y  appropriately,  j;iven  adequate 

opportunity  2 .0 

To  number  x  &  y  given  audible  encouragement  2.1 

To  find  x  &  v  always  2.1 

To  regroup  x  &  y,  given  audible  encouragement  2.2 

To  cross  out  x  &  y  at  least  three  out  of  four 

tin as,  given  pleasing  examples  2.5 

To  mark  x  &  y  satisfactorily  2.5 

To  learn  x  y  with  902  accuracy  2.6 

To  respond  to  x  &  y  with  no  more  than  two  errors, 
given  various  chances  2.7 

To  reject  x  6  y  2.7 

To  arrange  x  &  y,  given  highly  m<  tivating 

conditions  2.8 

To  borrow  from  x  &  y  consistently,  give.-,  pre¬ 
viously  unencountered  examples  2.8 

To  underline  x  &  y,  given,  examples  that  are 
aesthetically  satisfying  3.0 

To  average  x  &  y  understand ingly,  given  adequate 
opportunity  3  - 1 

To  connect  x  4  y  with  awareness,  given  a  cor¬ 
rectly  solved  example  for  reference  3.1 

To  analyse  x  6  y  on  the  first  sttem>t,  given 
carious  chances  3.2 


Variance 

0.0 

0.1 

0.1 

0.1 

0.2 

0.3 

0.7 

0.8 

0.4 

1.7 

0.7 

0.3 

0.8 

1.2 

1.5 


1.5 
2.0 

1.9 

1.3 

1.9 

1.7 

2.0 

2.0 

2.5 

1.5 

2.8 


1.5 

1.3 

2.3 
1.7 
2.1 
1.5 


32. 


Tab^e  2-1  (continued) 


33. 

To  read  z  &  y  orally  with  understanding 

3.2 

1  .9 

34. 

To  become  competent  In  x  4  y  within,  three  days: 

3.3 

2.1 

35. 

To  repent  %  b  y  orally  with  feeling,  given  highly 

stimulating  motivation 

3.4 

1.6 

36. 

To  choose  s  i  j  aesthetically,  given  certain  tools 

3  6 

1.5 

37. 

To  e  elect  x  6  y  satisfactorily 

5.6 

1.1 

38. 

To  complete  x  6  y  knowingly 

.6 

1.5 

39. 

To  finish  x  &  y  with  a war eo ess,  without  reference 

to  any  helps 

3.6 

2.0 

40. 

To  infer  x  6  y  knowingly,  given  five  statements 

of  fact  and  five  statements  of  opinion 

3.7 

1.1 

41. 

To  know  x  &  y  satisfactorily 

3.9 

1.3 

42. 

To  realize  x  &  y  fully,  given  some  opportunity 

4.0 

1.4 

43. 

To  perceive  x  &  y 

4.0 

1.8 

44. 

To  think  about  x  &  y  critically  at  least  three 

out  of  four  times  given  some  motivation 

4.1 

2.1 

45. 

To  think  about  x  6  y  with  1001  accuracy,  given  a 

choice  of  doing  so  or  xot 

4.3 

0.8 

46. 

To  feel  abemt  x  &  y  without  reference  to  any  aids 

4.3 

1.3 

47. 

To  understand  x  &  y 

4,4 

1.3 

43. 

To  concentrate  on  x  &  y,  given  no  more  than  ar. 

average  opportunity 

4.4 

1.3 

69. 

To  wonder  about  x  6  y,  given  previously  unencoun¬ 

tered  examples 

4.5 

0.6 

50. 

To  he  curious  about  x  &  y  always 

4.5 

0.5 

51. 

To  genuinely  appreciate  x  6  y,  given  a  choice  of 

doing  so  or  net 

4.5 

0.7 

52. 

To  be  appropriately  avate  of  x  &  t,  giver,  tradi¬ 

tional  encouragement 

4.6 

0.4 

53. 

To  like  x  6  y  genuinely,  given  generally  satisfac¬ 

tory  laotivation 

4.7 

0.2 

T  ' 


Table  2-2 

Intercorrelations  of  Mean  Ratings  Obtained  in  Study  2 


/ 


Criterion  Verb 


Complete  Statement 


Condition 

.26 

(n  •  24) 


.44  .78 

(n  »  22)  (n  »  38) 


on  statements  having  components  identical  fcc  those  used  in  the  first 
three  parts  of  the  questionnaire.  Table  2-3  gives  the  correlations  between 
a'd  among  the  mean  ratings  of  the  components  as  they  appeared  in  Part  IV. 

Discussion 

Although  the  study  supports  the  assumption  that  verbs  are  crucial  in 
statements  of  behavioral  objectives,  the  results  also  indicate  that  there 
is  more  to  constructing  an  objective  than  merely  selecting  a  "good"  verb. 

The  correlations  between  the  c  iplete  statements  and  the  components 
indicate  that  the  choice  of  criterion  statements  may  also  be  important. 

The  correlation  between  the  condition  and  criterion  may  be  partly  explained 
by  the  combinations  used  in  the  questionnaire.  Since  the  selection  of 
components  was  not  random,  l.e.,  the  resulting  statements  had  to  make 
sense,  some  bias  might  have  been  Introduced  into  the  study. 

Although  two  different  rating  scales,  observability  and  precision, 
were  used  in  the  questionnaire,  it  was  assumed  that  each  was  measuring 
the  same  characteristic  of  the  statement  and  that  the  ratings,  therefore, 
could  be  compared.  Obviously,  further  studies  were  needed  to  justify  this 
assumption. 

The  results  of  Study  2  indicate  that  when  statements  of  conditions 
or  statements  of  criteria  or  both  are  added  to  verbs  of  known  "observability," 
respondents'  perceptions  may  be  altered. 


Table  2-3 


Correlations  between  Ratings  of  Components  —  Study  2 


1. 

Condition 

Condition 

Criterion 

Verb 

2. 

Criterion 

.19 

-- 

-- 

1 

i 

3. 

Verb 

.05 

-.03 

-- 

i 

1 

V } 

4. 

Complete  Statement 

.16 

.49 

.78 

1 

! 

2 

3 

Study  3 


Since  **  had  obtained  evidence  to  the  effect  that  condition  and 
criterion  affect  raters*  perceptions  of  the  observability  of  an  objective, 
we  were  naturally  curious  as  to  whether  or  not  the  assumption  that  the 
verb  Is  the  most  Important  component  of  an  objective  can  be  supported. 
Consequently,  we  designed  a  study  to  determine  the  ratings  of  typical 
verbs,  conditions,  and  criteria  used  In  behavioral  objectives,  as  well  as 
ratings  of  complete  statements  of  objectives.  Additionally,  the  role 
which  each  type  of  component  plays  in  a  complete  objective  was  observed. 
Finally,  the  study  was  designed  to  explore  the  effects  of  t*e  dimension 
used  to  rate  the  various  expressions. 

Method 

Materials.  Twenty-five  objectives  were  created  by  combining,  on  a 
quasi-random  basis,  the  required  number  of  verbs,  conditions,  and  criteria. 
Some  statements  were  rearranged  to  avoid  combinations  that  seemed  eitner 
meaningless  or  frivolous  to  the  research  team  (e.g.,  "to  like...,  gl^en 
appropriate  tools").  The  components  were  selr:ted  to  cover  the  entire 
range  of  ratings  (1.0  to  5.0)  obtained  in  previous  work  (Oeno  and  Jerkins, 
1969;  Studies  1  and  2,  above).  The  rating  booklets  were  prepared  by 
randomly  listing  the  25  items  of  each  type  (verbs,  conditions,  criteria, 
objectives)  on  a  single  page  and  counterbalancing  the  order  of  the  pages 
to  control  for  order  effects.  Each  Item  appeared  once  in  Its  component 
list  and  once  in  the  list  of  objectives.  The  four  pages  of  lists, 
containing  the  100  items  to  be  rated,  plus  a  cover  page  giving  instructions, 
five  sample  ratings,  and  the  rating  dimension  to  be  used,  constituted  the 
experimental  booklet.  The  four  rating  dimensions  and  four  page  orders 


generated  sixteen  different  booklets. 

The  four  rating  scales  used  in  this  study  were  most  observable  - 
least  observable,  precise  -  vague,  clear  -  ambiguous,  and  concrete  - 
abstract.  Each  subject  rated  Items  on  only  one  of  these  scales.  This 
scale  was  presented  on  each  page  of  the  booklet,  along  with  a  diagram  on 
which  the  extremes  were  label  id  A  and  E.  Subjects  recorded  their  ratings 
on  IBM  scoring  sheets. 

Subjects  and  Procedure.  Eighty  students  In  an  Arizona  state  University 
upper-division  education  class  participated  in  this  study  as  part  of  a  regular 
class  session  The  booklets  were  Interleaved  in  sjch  a  manner  that  every 
sixteenth  subject  received  the  same  booklet.  The  subjects  were  given  oral 
and  written  instructions  explaining  the  method  of  nting  and  the  use  of  the 
scoring  sheet 

Data  Analysis.  This  study  was  designed  to  ascertain  the  correlations 
between  the  three  separate  components  of  behavioral  objectives  and  the 
complete  obje.'tive  in  which  they  are  embedded.  The  basic  data  consists 
of  the  means  ind  variances  of  the  subjects'  ratings  of  each  of  the  100 
Items  In  the  oooklet.  All  bivariate  coefficients  reperted  in  this  report 
are  Pearson  product  -  moment  correlation  coefficients. 

Analysis  was  done  by  computer,  using  appropriate  S’SS  programs,  after 
data  were  transferred  from  tne  IBM  answer  sheets  to  pnthed  cards. 

Resu i ts 

When  the  mean  ratings  for  the  objectives  are  compared  with  those  of 
the  components,  it  becomes  apparent  that  the  verb's  effect  on  the  rating 
is  not  as  great  as  had  been  assumed.  Table  3-1  gives  the  means  and 
variances  for  four  of  the  25  objectives  and  components.  Thes»  ratings 


Objective  #11 

1.5 

.7 

1.8 

1.4 

1.4  .8 

1,6 

To  measure 

1.8 

1.0 

1.6 

.8 

1.6  1.0 

S  1.3 

to  the  nearest  quarter  Inch 

2.0 

1.4 

1.8 

1.5 

1.4  .9 

1.4 

given  a  ruler 

2.0 

! 

1.7 

2.0 

2.4 

1.4  1.2 

1.8 

Objective  #18 

3.3 

1.0 

3.7 

1.2 

3.1  1.3 

3.6 

To  Infer 

3*4 

1.9 

3.6 

1.9 

3.4  1.5 

4.0 

given  previous  unencountered 

3.4 

1.4 

3.4 

1.9 

2.8  2.3 

3.2 

examples 

3.0 

1.3 

3.3 

2.2 

3.0  1.5 

2.7 

Objective  #13 

2.3 

2.0 

2.0 

1.8 

2.2  2.4 

2.2 

To  know 

3.0 

2.0 

3.7 

2.3 

3.6  1.1 

3.3 

In  the  same  order  as  that  given 

1.8 

1.3 

1.9 

1.4 

1.6  .8 

1.9 

In  the  textbook 

1 

given  an  unordered  list  of  Items 

2.6 

2.3 

2,8 

2.9 

2.2  1.6 

1 

2.0 

Objective  #2 

3,2 

1.5 

3.8 

1.2 

1 

2.8  1.9 

3.1 

To  read 

1.7 

.7 

1.8 

1.4 

1.9  1.4 

1.8 

with  feeling 

2.7 

1.9 

4.2 

1.6 

2.2  .8 

3.9 

given  a  passage  not  encountered 

3.1 

2.0 

3.6 

2.2 

2.2  1.8 

2.6 

In  the  previous  week 

1.6 


1 . 

1. 

2. 

2. 


1.7 


3.4 

1.1 


1.1 


2.0 


.9 

1.1 

1.4 


i 
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arc  fairly  consistent,  across  the  four  rating  scales.  In  the  first  two 
objectives,  #11  and  #18,  little  difference  Is  found  when  the  mean 
rating  for  the  objective  is  compared  with  the  verb  rating.  However,  when 
similar  comparisons  are  made  for  the  last  objectives,  #15  and  #2,  distinct 
differences  are  observed.  In  these  latter  cases,  the  ratings  for  the 
conditions  and  criteria  are  different  from  those  of  the  verbs  and  the 
ratings  for  the  objectives  tend  to  agree  more  with  those  obtained  fo? 
the  conditions  and  criteria. 

The  correlations  between  mean  ratings  of  individual  components  and 
objectives  are  shown  In  Table  3-2.  In  each  case,  the  correlation  Is  based 
on  25  pairs  of  scores  —  25  component  means  and  25  objectives.  The  pattern 
of  intercorrelations  Is  generally  the  same  for  each  rating  scale,  and  the 
relative  homogeneity  of  the  coefficients  suggests  that  each  component 
makes  nearly  the  same  contribution  to  the  rating  of  the  objective. 

Positive  correlations  were  also  found  between  the  mean  ratings  of 
conditions  and  criteria.  These  correlations  ranged  from  +  .14  to  +.36  and 
do  not  overlap  with  those  reported  In  Table  3-2,  indicating  that  we  are 
dealing  with  very  discrete  and  different  phenomena. 

Discussion 

The  results  of  this  study  clearly  indicate  that  conditions  and 
criteria  contribute  to  Individuals'  perceptions  of  objectives.  This  is 
In  marked  contrast  to  findings  of  earlier  studies,  where  the  emphasis  was 
on  the  verb.  The  present  study  provides  strong  Indication  that  variables 
other  than  the  observability  of  the  verb  must  be  considered  when  objectives 
are  constructed.  Instructional  designers,  researchers,  and  evaluators 
must  also  be  concerned  with  the  choice  of  conditions  and  criteria. 


Table  3-2 

Correlations  between  Ratings  of  Components 
and  Complete  Objectives  —  Study  3 


Rating  Scale 

Verb 

Condition 

Criterion 

Observability 

.43 

.60 

.58 

Precision 

.39 

.59 

.71 

Clarity 

.42 

.62 

.53 

Concreteness 

.56 

.52 

.53 
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Since  It  Is  often  meaningless  to  consider  the  observability  of 
conditions  and  criteria  In  Isolation  (e.g.,  "given  the  occasion,"  or 
"confidently" },  the  similarity  of  ratings  for  the  four  scales*- 
observability,  precision,  clarity,  and  concreteness—  Indicate  that 
these  characteristics  could  be  used  Interchangeably.  If  an  objective  Is 
to  be  used  to  guide  a  student's  study,  It  may  be  more  Informative  to  talk 
about  Its  precision  or  clarity  than  about  Its  observability.  In  such 
cases,  the  Instructional  designer  wants  to  be  certain  the  student  knows 
what  Is  Important  in  the  material.  This  Is  In  contrast  to  the  teacher's 
desire  to  know  how  the  student  will  respond  when  he  has  mastered  the 
material.  It  Is  In  the  latter  case  that  an  observable  behavior  becomes 
Important. 

The  quasi -random  procedure  used  to  assemble  the  objectives  may  have 
resulted  In  a  "built-in"  correlation  between  the  conditions  and  criteria 
In  some  objectives.  The  investigators  edited  the  list  In  an  effort  to 
make  sensible  objectives.  Nonsensical  combinations,  such  as  "To  construct 
x  genuinely,  given  a  passage  not  encountered  during  the  past  week"  were 
either  edited  or  replaced.  This  probably  Introduced  a  bias  against 
objectives  In  which  very  vague  criteria  were  coupled  with  very  precise 
conditions  or  In  which  very  precise  criteria  were  coupled  with  very  vague 
conditions.  However,  as  far  as  method  Is  concerned,  this  was  probably  the 
best  way  to  proceed  In  order  to  avoid  meaningless  objectives  and  to  simulate 
the  realistic  use  of  objectives  within  an  Instructional  milieu. 

One  other  factor  may  have  Influenced  the  ratings  obtained  In  this 
study.  In  the  list  of  objectives,  "x"  and  "y"  were  Inserted  for  the  direct 
objects  to  avoid  the  use  of  specific  subject  matter  In  the  study.  The 
respondents'  "mental"  choice  of  a  replacement  for  these  components  may  also 
have  Influenced  their  ratings. 
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Study  4 

Our  next  effort  in  developing  an  operational  definition  of  the 

concept  "performance  objective"  stems  directly  from  Study  3.  There  we 

found  that  a  substantial  portion  of  the  variance  of  ratings  of  total 

objectives  could  be  accounted  for  by  a  linear  combination  of  the  three 

component  parts,  verb,  condition,  and  criterion.  The  variance  unaccounted 

for  was  a  nagging  concern,  however,  and  led  to  a  reanalysis  of  the  problem. 

In  the  reanalysis.  It  became  clear  that  the  role  of  the  direct  object  was 

being  Ignored.  In  all  studies  up  to  this  point,  "x"  and  "y"  had  been 

Inserted  where  the  direct  object  would  normally  be  found,  e.g.,  "to  draw 

x  accurately  from  memory."  This  was  done  to  avoid  Interference  from  choice 

of  specific  subject  matter;  In  many  practical  cases,  the  direct  object 

constitutes  a  "given"  that  Is  ordinarily  not  subject  to  variation  by  the 

# 

Instructional  designer.  To  the  degree  that  subjects  mentally  Inserted 
their  own  choice  of  direct  object,  the  overall  rating  may  have  been 
affected.  For  example,  a  subject  who  thinks  In  terms  of  fairly  concrete 
objects  such  as  "a  square"  sight  rate  the  overall  objective  as  much  more 
precise  and  observable  than  one  who  mentally  inserts  the  word  "something." 
Furthermore,  it  seemed  possible  that  the  use  of  the  unspecified,  abstract 
letters  as  direct,  objects  might  have  caused  a  general  shift  toward  perception 
of  objectives  as  less  observable  or  less  precise.  For  these  reasons,  it 
seemed  desirable  to  determine  the  strength  of  association  between  the  choice 
of  direct  object  end  the  overall  rating  of  the  complete  objective. 

The  fourth  study  was  designed  to  explore  the  role  of  the  direct 
object  In  the  perception  of  the  complete  behavioral  objective.  The  study 
was  designed  to  answer  the  following  specific  questions: 
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(1)  Does  the  Inclusion  of  direct  objects,  as  opposed  to  abstract 
direct  objects  (l.e. ,  "xM  or  Hy"),  In  statements  of 
behavioral  objectives  generate  an  overall  shift  of  ratings 
in  the  direction  of  increased  observability  and  precision? 

(2)  Do  the  ratings  of  Individual  objectives  change  as  a  result 
of  the  choice  of  different  direct  objects? 

Since  It  could  be  argued  that  a  positive  answer  to  the  second 
question  would  cloud  the  interpretation  of  a  positive  answer  to  the  first, 
tfe  lists  used  were  balanced  by  selecting  direct  objects  that  covered  a 
wide  range  cf  observability.  In  this  way,  It  was  hoped  to  avoid  the 
objection  that  any  overall  shift  was  caused  by  using  highly  observable 
direct  obje:ts  throughout. 

Method 

Subjects  and  design.  Sixty-four  undergraduate  students  from  two 
Arizona  Suite  University  upper  division  Instructional  media  courses 
participated  in  this  study  as  part  of  their  regular  class  sessions.  Four 
additional  students  omitted  Items  from  their  response  booklets,  rendering 
these  borklets  useless;  their  data  were  excluded  from  the  study. 

Responses  of  44  of  these  subjects  were  used  In  the  main  design  of  the 
study.  Each  of  these  subjects  rated  24  objective  statements  containing  24 
different  direct  objects.  Each  subject  also  rated  the  24  direct  objects 
In  Isolation  for  purposes  of  comparison.  The  24  objects  were  organized 
Into  six  objective  sets  of  four  objectives  each.  Within  each  set,  the 
verb,  condition,  and  criterion  were  Identical,  so  that  the  four  objectives 
varied  only  In  choice  of  direct  object.  Different  sets  used  different 
verbs,  conditions,  and  criteria,  and  no  component  of  an  objective  was  used 
In  two  different  objective  sets.  The  arrangement  of  components  is  shown 
in  Table  4-1,  from  which  the  entire  set  of  24  objectives  can  be  reconstructed. 


Table  4-1 

Components  of  the  Six  Objective  Sets  -  Study  4 
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The  experimental  design  was  a  repeated  measures  nested  design,  with  six 
objective  sets  and  (four  different)  direct  objects  nested  within  objective 
sets.  This  design  was  replicated  with  two  rating  dimensions,  observability 
and  precision;  22  subjects  were  used  for  each  rating  dimension.  The  ratings 
of  direct  objects  In  isolation  were  collected  for  the  purpose  of  determining 
the  correlation  between  direct  objects  and  complete  objectives  containing 
those  direct  objects. 

In  addition  to  the  main  design,  20  additional  subjects  rated  six 
complete  objectives  containing  "x"  or  "yH  In  place  of  the  direct  object. 
These  six  objectives  corresponded  to  the  six  objective  sets  used  in  the 
main  design.  These  ratings  were  collected  to  compare  overall  ratings  with 
and  without  explicit  direct  objects.  Again,  half  the  subjects  rated 
observability  and  half  precision. 

Materials  and  Procedure 

Subjects  in  the  main  design  each  received  a  booklet  containing  four 
pages  of  complete  objectives,  six  to  a  page.  On  each  page,  one  objective 
from  each  objoctlve  set  appeared.  A  fifth  page  contained  the  24  direct 
objects  to  be  rated  In  Isolation.  The  rating  scale  to  be  used  was  printed 
at  the  top  of  each  page  In  the  booklet.  In  a  diagram  depicting  the  full 
range  of  the  scale,  most  observable  (1)  to  least  observable  (5)  or  precise 
(1)  to  vague  (S).  The  same  rating  dimension  was  used  throughout  each 
booklet.  Subjects  recorded  their  responses  directly  in  the  booklets.  The 
booklets  were  assembled  In  counterbalanced  order,  to  avoid  order  effects, 
and  were  stapled  to  a  cover  page  which  provided  instructions  and  two 
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sample  rar1ngs.* 

For  the  20  subjects  rating  objectives  without  direct  objects,  the 
six  objectives  to  be  rated  were  all  on  a  single  page;  the  same  cover  sheet 
was  used  for  these  booklets. 

The  24  direct  objects  used  in  this  study  were  selected  from 
objectives  developed  by  the  National  Assessment  Project  (1972).  The  pool 
of  direct  objects  was  rated  a  priori  by  the  experimenters  for  observability 
and  precision;  the  final  24  selected  provided  a  wide  range  of  values  within 
each  objective  set.  The  six  verts  used  (cf.  Table  4-1)  were  selected  from  a 
pool  of  verbs  that  are  not  domain  specific,  and  for  which  earlier  studies 
had  provided  data  concerning  the  perceived  observability,  both  In  isolation 
and  in  context.  They  were  chosen  so  that  the  mean  ratings  covered  the 
entire  spectrum  of  the  observability  scale.  The  conditions  and  criteria 
we^e  chosen  in  similar  fashion.  The  components  were  combined  Into 
objectives  In  such  a  way  as  to  avoid  absurdities,  and  to  be  consistent 
with  generally  accepted  grammatical  and  contextual  conventions. 

The  subjects  were  given  oral  and  written  instructions  by  the 
experimenter  at  the  outset,  along  with  the  examples  explaining  the  method 
of  rating.  Specific  reference  was  made  in  the  instructions  to  the  fact 
that  these  statements  are  typical  of  expressions  used  in  instructional 
objectives.  The  subjects  were  also  instructed  not  to  refer  back  to  a  page 
once  it  had  been  completed. 


♦All  subjects  also  rated  25  verbs,  25  conditions,  and  25  criteria 
as  part  of  a  replication  of  previous  work.  The  results  from  these  ratings 
generally  confirm  previous  results,  and  they  will  not  be  discussed  further 
in  this  report. 
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Resul ts 

The  means  and  standard  deviations  for  objective  sets,  with  and 
without  direct  objects,  are  shown  In  Table  4-2  separately  for  each  rating 
dimension.  It  Is  readily  apparent  that  there  Is  no  significant  difference 
between  subjects'  ratings  of  objectives  with  direct  objects  and  objectives 
containing  ax"  and  "y."  Tils  Is  reflected  in  tests  computed  separately  for 
observability,  t  (30)  *  0.60,  and  precision,  t  (30)  ■  1.18. 

Turning  to  the  quesclon  of  differences  induced  by  choice  of  direct 
object,  ah  analysis  of  variance  was  computed  separately  for  each  of  the 
two  rating  dimensions;  this  was  done  for  purposes  of  both  simplicity  and 
clarity.  The  results  cf  the  two  analyses  are  shown  In  Tables  4-3  and  4-4. 

As  expected,  since  an  effort  had  been  made  to  construct  objective  sets 
differing  In  ratings,  the  mean  ratings  for  verb  sets  differed  significantly; 
In  the  observability  analysis,  the  most  observabl*  objective  was  "to 
accurately  draw  *  graph  from  memory"  and  the  least  observable  objective 
was  "to  understand  computer  processes  without  error,  given  audible 
encouragement";  in  the  precision  analysis,  the  most  precise  objective 
was  "to  draw  slaple  floor  plans  nf  furnishings  in  rooms,  from  memory' 
and  the  least  precise  was  "to  appropriately  Identify  stated  and  unstated 
assumptions,  given  various  chances."  More  Importart  was  the  questio  of 
whether  the  direct  objects  caused  significant  differences  within 
objective  sets,  that  is,  whether  the  four  objectives  within  objective 
sets  differed.  The  results  are  clear  for  both  ratiig  dimens4 :.is.  The 
objectives  within  sets  differ  significantly  more  tha»  would  be  expected 
by  chance;  for  observability,  F  (18,378)  ■  2.41,  £  <  .01;  for  precision, 

F  118,378)  ■  2.27,  £  <  .01.  This  finding  is  further  bo'stered  by  the 
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Table  4-2 

Means  and  Standard  Deviations  of  Ratings  for  Objective  Sets  —  Study  4 


Observability 

Precision 

l 

S 

N 

3T 

S 

N 

Objectives  containing 
x  and  y  in  place  of 

direct  objects 

Objectives  with  direct 

3.012 

.4754 

10 

3.<&6 

.  .6164 

10 

objects 

2.890 

.5385 

22 

2.760 

.6132 

22 
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Table  4-3 

Analysis  of  Variance  Seminary  Table  for  Observability  —  Study  4 


Source 

SS 

Subjects 

145.9773 

Verb  Sets 

119.197 

Direct  Objects  within 
Verb  Sets 

40.8636 

Subjects  by  Verb  Sets 

303.3863 

Residual 

356.6364 

TOTAL 

966.0606 

df 

MS 

F 

21 

6.951 

5 

23.839 

10.502* 

18 

2.270 

2.4059* 

105 

2.8894 

378 

*p  <  .01 
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Table  4-4 

Analysis  of  Variance  Summary  Table  for  precision  —  Study  4 


Source 

SS 

df 

K5 

F 

Subjects 

189.3106 

21 

9.0148 

Verb  Sets 

47.3333 

5 

9.4666 

5.28* 

Direct  Objects  within 

Verb  Sets 

32.2955 

18 

1.7942 

2.27* 

Subjects  by  Verb  Sets 

261.6667 

105 

2.4921 

Residual 

299.2045 

378 

.7915 

TOTAL 

829.8106 

*p  <  .01 
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correlations  between  ratings  of  direct  objects  and  ratings  of  objectives 
containing  those  direct  objects,  computed  within  sets  and  averaged:  +.78 
for  observability  ratings  and  +.41  for  precision  (g/s  <  .05). 

Discussion 

The  resuits  indicate  clearly  that  the  choice  of  direct  object  does 
influence  raters'  perceptions  of  the  observability  and  precision  of  a 
behavioral  objective.  Taken  together  with  the  results  of  Study  3,  these 
results  provide  convincing  evidence  that  no  single  component,  such  as  the 
ven,  should  bu  singled  out  as  being  of  primary  Importance  in  determining 
thf  character  of  a  behavioral  objective.  Rather,  careful  attention  must 
be  paid  to  ail  components  to  insure  an  objective  that  is  observable, 
mdse,  clear,  and  concrete.  This  conclusion  must  be  tempered  by  the 
realization  that  the  Instructional  designer  does  not  have  complete 
freedom  i»  selecting  direct  objects.  Instead,  they  are  often  specified  by 
the  user  organization  as  part  of  the  nstructional  goal.  For  example, 
if  the  aim  Is  to  teach  the  student  to  multiply  fractions,  there  Is  little 
opportunity  to  substitute  another  direct  object.  However,  once  alerted 
to  the  problem,  the  instructional  designer  will  have  no  difficulty,  having 
recognized  the  inherent  imprecision  of  isking  the  child  to  "draw  a  nice 
picture,"  in  seeking  a  more  precise,  observable  direct  object. 

The  absence  of  a  pronounced  shift  oward  greater  observability  and 
precision  for  objectives  containing  dlnjct  objects  suggests  that  subjects 
either  (1)  substitute  mentally  their  own  direct  objects,  which  average 
out  to  about  tu  same  values  as  real  direct  objects,  and/o”  (2)  ignore  the 
"x"  as  an  acti  e  element  in  the  objectl /e  when  they  have  not  been  exposed 
to  other  sentetces  containing  real  direct  objects.  Regardless  of  the 


Interpretation,  the  lack  of  a  radical  shift  lends  increased  confidence  to 
results  obtained  in  previous  studies  employing  only  "x"  and  "y"  In  objectives. 

The  empirical  data  gathered  In  this  study  on  the  ratings  of  the  components 
of  a  behavioral  objective  In  isolation  and  within  complete  statements  of 
objectives  indicate  that  the  selection  of  the  various  components  influences 
individuals'  perceptions  of  objectives.  The  data  suggest  that  we  are  moving 
closer  to  a  consistent  operational  definition  of  the  behavioral  objective. 

As  the  function  of  the  individual  components  becomes  more  clearly  delineated, 
educators  and  trainers  should  be  able  to  select  and  to  construct  more  precise 
statements  of  performance  objectives. 

Study  5 

Thus  far  our  studies  have  led  us  to  conclude  that  emphasis  on  any  one 
component  of  a  performance  objective  is  unwarranted.  This  finding  is 
Inconsistent  with  the  injunctions  and  exhortations  found  In  most,  if  not 
all,  manuals  and  guidelines  for  instructional  systems  design.  Our  studies 
clearly  indicate  that  empirical  evidence  Is  needed  to  determine  how  an 
objective  should  be  constructed  In  order  to  maximize  the  possibility  that 
the  reader  will  perceive  It  as  describing  an  observable  event. 

However,  all  the  studies  cited  above  used  educators  or  students  in 
colleges  of  education  as  subjects.  Generalizing  the  findings  to  other 
populations  would  be  warranted  only  if  a  high  relationship  could  be  found 
between  the  results  of  the  studies  cited  and  the  results  of  a  replication 
using  another  population.  Specifically,  It  was  hypothesized  that  the 
findings  could  be  generalized  to  two  types  of  military  personnel:  trainees 
and  trainers  (including  Instructional  systems  designers). 
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To  gather  preliminary  data  for  testing  this  hypothesis,  replications  of  Study 
1  were  conducted  on  two  target  populations.  We  felt  this  step  was  required 
because  no  other  historical  data  were  available  which  could  be  used  as  a 
basis  for  comparison  with  the  Deno-Jenklns  results. 

The  first  replication  used  30  undergraduate  pilot  trainees  at  Williams 
AFB,  Arizona.  They  were  administered  an  instrument  containing  123  verbs 
which  each  trainee  was  asked  to  rate  on  a  quintile  scale  from  most  observable 
to  least  observable.  This  instrument  was  exactly  the  same  as  that  used  by 
Deno  and  Jenkins  (1969),  with  one  addition:  the  previous  Instrument 
contained  99  verbs  found  In  school  curricula*,  the  present  Instrument  contained 
all  these  plus  24  verbs  found  In  USAF  Undergraduate  Pilot  Training  syllabi.  A 
second  replication,  using  the  same  Instrument,  was  conducted  at  luke  AFB, 
Arizona,  where  17  members  of  four  Instructional  Systems  Development  teams 
served  as  subjects.  „ 

Results 

The  correlations  between  the  ratings  of  the  pilot  trainees  and  the 
results  reported  by  Deno  and  Jenkins  was  .90.  The  correlation  between  the 
ratings  of  the  ISO  team  members  and  the  Deno-Jenklns  results  was  .91.  The 
correlation  between  the  trainee  and  ISO  ratings  was  .89. 

The  mean  rating  of  the  99  verbs  used  by  Oeno  and  Jenkins  was  3.06. 

Pilot  trainees  gave  these  same  verbs  a  mean  rating  of  2.57,  while  ISO  team 
members  rated  them  2.51.  The  24  verbs  which  were  selected  from  ATC  training 
materials  were  given  mean  ratings  of  2.46  and  2.50  by  the  UPTs  and  ISO  team 
members,  respectively. 
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Conclusions 

The  correlations  ootained  are  sufficiently  high  tc  warrant  our 
concluding  that,  with  respect  to  the  content  of  this  study,  we  are  dealing 
with  highly  similar  if  not  identical  populations.  Consequently,  we  tentatively 
inferred  that  the  findings  of  earlier  studies  concerning  the  three  components 
of  performance  objectives,  as  well  as  the  findings  concerning  direct  objects, 
are  generalizable  to  the  context  of  Air  Force  training. 

It  appears  that  some  of  the  operational  difficulties  currently 
encountered  by  field  ISO  teams  (especially  for  complex  perceptual -motor  skill 
training)  would  be  reduced  If  existing  manuals  were  revised  and  new  manuals 
written  to  incorporate  the  findings  concerning  the  observability  of  verbs. 
Furthermore,  the  three  factor  model  of  objectives  (verb,  conditions,  and 
criteria)  should  be  expanded  to  Include  the  direct  object. 

Study  6 

Having  established  a  precise  form  from  which  objectives  should  be  drawn 
In  the  above  five  studies,  the  final  study  was  conducted  to  examine  how  well 
they  would  function  in  the  classroom.  In  the  earlier  reported  research, 
little  support  was  found  for  the  training  of  teachers  in  the  use  of  behavioral 
objectives.  In  addition,  the  effectiveness  of  teacher  training  was  measured 
by  subsequent  student  performance  rather  than  the  resulting  behavior  of  the 
teacher.  To  overcome  this  obvious  methodological  flaw,  a  technique  was 
developed  whereby  the  instructor's  performance  in  the  use  of  objectives 
was  independently  evaluated  by  a  trained  observer.  Thus,  a  sixth  study  was 
conducted  to  assess  —  through  experimental  isolation  of  the  instructor 
training  variable  --  both  the  effects  of  training  in  the  use  of  behavioral 
objectives  on  teacher  behavior  and  student  achievement. 


Method 


PH or  to  the  experiment,  19  graduate  teaching  assistants  were  administered 
a  pretest  that  assessed  their  a  priori  knowledge  level  on  the  use  of  behavioral 
objectives.  After  subjects'  stratification  by  high  and  low  a  priori  knowledge 
level,  they  were  randomly  assigned  to  either  a  training  or  no-training 
treatment  condition.  The  subjects  assigned  to  the  training  condition  received 
instruction  on  the  use  of  behavioral  objectives  and  a  posttest  assessing  their 
cognitive  performance  in  acquiring  the  teaching  skills.  All  subjects  then 
received  the  instructors'  materials  and  directions  for  teaching  a  short  unit 
of  instruction.  Subjects  were  assigned  to  an  instructional  setting  arid 
informed  that  they  should  present  the  short  unit  of  instruction.  Prior  to 
the  instructional  sessions,  10  observers  were  trained  to  rate  on  an  observation 
form  the  quality  of  the  Instructor's  behavior  (l.e.,  the  degree  of  use  of  the 
behavioral  objectives  of  the  unit)  on  a  scale  ranging  from  vague  (1)  to  precise 
(5).  After  each  Instructional  session,  a  post-teaching  pupil  assessment  was 
administered. 

Design.  The  basic  design  was  a  posttest-only  2x2  factorial  design 
with  Student  Achievement  as  the  dependent  variable  and  the  variables  Training 
level  and  A  priori  knowledge  level  as  the  Independent  variables. 

Subjects .  Subjects  were  19  volunteer  graduate  teaching  assistants, 
who  served  as  Instructors,  and  170  undergraduate  student  volunteers  from 
Arizona  State  University  who  received  the  instruction  and  completed  the 
posttest.  Students  were  randomly  assigned  to  Instructional  groups. 

Materials.  The  pretest  administered  to  the  instructors  to  assess 
their  a  priori  knowledge  level  on  the  use  of  objectives  consisted  of 
items  that  required  the  subject  to  distinguish  between  objectives  and 
activities,  to  identify  worthwhile  objectives,  and  to  Identify  well-written 
objectives.  The  Instructors'  training  consisted  of  the  following:  a 
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self-instructional  booklet  on  objectives  developed  for  and  tested  in 
the  Competency-Based  Instruction  course  In  the  Department  of  Educational 
Technology  at  Arizona  State  University;  practice  exercises  and  feedback 
for  each  objective  covered  In  the  booklet  (to  distinguish  between  activities 
and  behavioral  objectives,  to  Identify  well-written  objectives);  and  a 
30-Item  examination  over  the  objectives.  The  Instructors'  material  —  an 
11  page  handout  —  consisted  of  a  short  unit  on  the  Innovation  decision¬ 
making  process  model.  The  material  Included:  (a)  a  general  Introduction, 
(b)  four  behavioral  objectives,  (c)  a  discussion  of  the  four  stages  of  the 
model,  (d)  a  graphic  model  of  the  four  stages,  and  (e)  an  applied  example 
of  each  of  the  four  stages  of  the  model.  Ten  graduate  assistants  were 
trained  to  serve  as  classroom  observers  and  to  rate  the  Instructors' 
performances.  They  were  given  the  following  materials:  an  abbreviated 
form  of  the  Instructors'  material,  consisting  of  the  (a)  general  Introduce 
tlon,  (b)  the  four  behavioral  objectives,  (c)  a  graphic  model  of  the  four 
stages,  and  (d)  an  applied  example  of  each  of  the  four  stages  of  the  moosl. 
Each  student  received  an  objective  posttest  that  consisted  of  10  multiple 
choice  and  short  answer  Items  matched  to  the  four  objectives  of  the 
Instructional  unit. 

The  observers  were  given  an  observation  form  containing  12  statements 
about  the  vagueness  and  preciseness  of  the  Instructors'  behavior  (i.e.,  the 
use  of  the  behavioral  objectives  of  the  instructional  unit). 

Instructors'  training.  Results  of  a  pretest  on  the  use  of  behavioral 
objectives  were  used  to  stratify  subjects  according  to  high  and  low  a  priori 
knowledge  levels.  Subjects  were  then  randomly  assigned  to  one  of  two 
experimental  conditions.  The  10  subjects  assigned  to  the  training  condition 


completed  a  self-instructional  booklet  on  the  use  of  behavioral  objectives, 
practice  exercises,  and  a  30  item  posttest.  Then  these  subjects  were  given 
the  Instructors'  materials  and  the  following  instructions:  Instructions 
will  be  delivered  In  a  classroom  setting;  use  visual  aids,  handouts,  or  any 
other  Instructional  material  during  the  Instructional  sessions;  do  not 
exceed  30  minutes;  students  will  be  given  a  short  examination  on  the 
Instructional  unit.  The  Instructors  were  not  Informed  of  the  observers  and 
the  ratings  on  the  observation  forms  prior  to  Instruction.  The  nine  subjects 
assigned  to  the  no-training  group  condition  received  the  same  instructors' 
material  and  Instructions  cs  the  subjects  In  the  training  condition.  For 
both  conditions,  each  subject  was  Informed  when  and  where  to  meet  for  a 
30  minute  Instructional  session. 

Observers'  training.  After  receiving  the  abbreviated  form  of  the 
Instructional  materials,  the  observers  were  asked  to  familiarize  themselves 
with  the  content  of  the  instructional  unit  to  be  presented.  After  the 
Instructional  session,  the  observers  rated  on  an  observation  form  the 
Instructor's  behavior  on  a  scale  ranging  from  vague  (1)  to  precise  (5). 

All  responses  on  the  observation  form  were  summed  to  yield  a  total  score 
per  observer;  this  constituted  the  operationalization  of  the  Observation 
variable. 

Experimental  session.  Prior  to  each  Instructional  session,  students 
were  asked  by  the  experimenter  for  their  attention  and  participation  for 
the  presentation  by  the  Instructor.  The  students  were  informed  that  they 
would  be  asked  to  answer  questions  on  an  examination  form  about  the  Instruc¬ 
tional  material.  Following  the  Instructional  section,  students  completed  a 
short  examination  and  were  dismissed. 


Result 


Posttests  were  scored  for  number  correct.  *n  ara i/sis  vs-'. 
revealed  no  significant  differences  on  studert  achieve"^  t  i’jMe  t  .  'C*n, 

data  generated  by  the  observer  ratings  yielded  several  cignl*"-': a.,-,: 
important  effects,-  The  mean  amount  of  Students’  Ac  ir  eve-rent  corrv.ia"'-! 
positively  with  tht  Observations  by  the  observers  the  Trr'ri-i^  a- a  \; 
Training  conditions.  The  corresponding  r_  for  the  Training  condition  was 
.64,  2.  <  *201.  After  the  Observation  scores  for  both  conditions  wer<* 
divided  into  High  and  Low  Observation  scores  through  the  n-solit  technique, 
it  was  found  that  in  the  Training  and  Me  Training  conditions  a  consistently 
high,  positive  correlation  was  found  between  the  war  Students'  Achievement 
scores  for  each  subject  (i.e.  instructor)  and  the  total  Observation  '.core 
for  each  subject:  for  the  High  Training  group  r  *  .P7,  o  ».  ,  fcr  iht. 

Low  Training  group  r  *  .73,  p  <  .06;  fcr  the  High  No  draining  gruw.i  >  • 

.77,  p  <  .05;  for  the  Low  No  Training  group  _r  -  .84,  £  <  .05., 

Finally,  all  Observation  scores  were  grouped  into  High  inci  core, 
through  the  n-snlit  technique.  These  groupings  were  correlated  wit*  the 
corresponding  means  of  the  Students’  Achievement  scores,  regardless  of  tno 
Training  or  No  Training  conditions.  For  the  High  reservation  group  y  1 
.52,  £  <  .05;  for  the  Low  Observation  group  r  3  .75,  £  *'  .91.  Tnese 
correlations  indicate  that  the  observations  and  the  corresponding  students* 
achievement  have  a  high  degree  of  ctatistical  association 

Conclusions 

The  hypothesis,  students  instructed  by  teachers  trained  in  the  use 
of  behavioral  objectives  would  achieve  higher  scores  cn  an  examination  than 
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Table  6-1 

Anova  Table  for  Study  6 


Source 

SS 

DF 

MS 

F 

A 

Treatments 
(between  training  & 
no  training  grouus) 

3.112 

1 

3.512 

.53 

B 

a  priori  knowledge 
Tbetween  all  Highs  & 
Lows) 

7.S02 

1 

7.902 

1.20 

Interaction 

(»XB) 

17.780 

1 

17.780 

2.70 

Error 

1053.854 

160 

6.587 

6C 


students  instructed  by  untrained  teachers,  was  not  supported  in  tnis  study. 
The  results,  the  lack  of  significant  differences  between  the  Training  and 
No  Training  conditions,  support  the  earlier  findings  of  Cardarelli  (1971) 
and  Clingman  (1972),  who  failed  to  find  a  facultative  effect  of  the 
training  of  teachers  in  the  use  of  behavioral  objectives  as  reflected  by 
students'  achievement.  The  results  of  the  within  group  analyses,  i.e., 
the  variances  within  the  Training  and  No  Training  conditions,  suggest  that 
a  priori  knowledge  of  the  use  of  behavioral  objectives  does  not  have  an 
effect  on  students'  achievement. 

Although  significant  differences  were  not  yielded  by  this  study,  the 
experimental  isolation  of  the  Training  variable  and  the  use  of  the  classroom 
observers  and  their  resulting  observation  scores  for  each  subject  provided 
a  new  dimension  to  the  research  on  the  facilltative  effects  of  training 
teachers  in  the  use  of  behavioral  objectives.  The  Mgn  correlations  between 
the  Observation  scores  and  the  mean  Students'  Achievement  scores  for  each 
subject  suggest  that  the  training  may  have  influenced  the  classroom  behavior 
of  the  ‘nstructors,  but  this  was  not  reflected  in  the  students'  achievement 
scores.  In  any  case,  it  is  clear  that  when  teachers  behave  as  though  they 
are  using  objective-based  instruction,  student  achievement  improves.  Stated 
another  way,  if  the  parameters  set  by  objective-based  instruction  are 
followed  in  teacher  behavior,  more  learnirg  appears  to  occur.  The  difficulty 
in  demonstrating  this  effect  experimentally  lies  in  the  fact  that  human 
behavior  cannot  easily  be  manipulated,  especially  in  short-term  controlled 
settings.  Further  study  is  needed  in  identifying  these  " fad litative" 
behaviors,  exploring  whether  and  how  they  can  be  promoted,  and,  finally, 
whether  or  not  they  do  in  fact  improve  instruction.  It  is  critical  that 
we  fully  describe  the  factor  before  we  attribute  to  it  an  effect. 


I’-jre r  /  an;  Jorcl  us ^  on  > 


The  f  re;  err  "  ;x  r\  *«■,  found  z^'.  vii’e  coi  tempo  atj  ins*ri,c  .  ir;  v 
■> . '  tens  a.'S,  ,re  t*  s-  e  >'■'  >ome  ‘orm  c*  ,.erforrrt  ice  rt  ect-ves,  «.  mp  j  ri  c  a  ; 

S  ipo-i  tu  !iSt'.fv  ivei  *  j3  is  1 3  c The  vist  » u-^be*-  of  ituciies  on 
thi.  tcpv  da.orM -• ;?  &  ••  : ; ’*ad  s c to. _■/  tr -ee-way  s p *  r' aroic  -  '.hie^erent 
mcreirifn  ,  it crt ;  art,  ."{  (frect  .nowever,  the  int-itiv*  :<•?$'  rat 

oi  object'  v  -3f  an.:  -  erf  1 1  >;  s  pcsHivt  infective  data  h-'-  orcud-d  otje.'.nuS 

the  support  :hey  v-etf  io?  m*'-iLed 

The  >estar:h  nd  rfeiona!  .--.verc  i sc  contained  in  th’S  report  offer-  ,\ 
pai  to  a  1  pxpl,  nation  of  #ny  such  confusing  results  h.  v.-  1  ague  J  extorts  to 

deiPcnstrahe  <i oe  e '  >ot  o*  i;  Kctivo' .  tt  was  ^ i rs t  cor. tended  that  neither 
the  forctcn  nor  ;m-  f-  or  of  pe^formanc?  objective  hove  tea  operational ! / 
defined.  Of  the  ott.Jie  >  . -r;  irwed,  a  mol tuous  oefiri-;  ons  eliminated  trie 
rocsitil’ty  of  rep-icaN'or  th  even  mi  no**  ge,  e-aii  ratio*,  The  dependent 
•measures  v?r!ird  widely,  caigln-.  f>'o:r.  teacher  ochanor  tr.  student  affect 
and  achieve iv  nt ,  live 'ting  cfe  possibility  cf  any  practice.!  conclusion-',  'sa/oj 
those  regard  rig  ro-.ear :  i  cethc>ds ) .  Thus,  research  nas  yn  to  pro-/;  io  an 
empirita1  Canon*  t  tit* or  ot  :b*ectives'  f.ncticr.. 

As  to  U'  ‘c.ni  of  objectives,  all  agree  that  it  is  'ecessary  to  state- 
explicit  behaviors,  condition*,  and  criteria  { standards  > .  Argurnuiy  begins 
in  deteraini  13  rha:  is  considered  exo licit.  -‘.t searchers  address'ng  this 
issue  have  acte-rptec!  to  compile  "recommended  lists"  of  verbs  to  he  used  in 
objectives,  and  it  is  this  hind  of  data  whict  ;ro.iaed  a  sprngbuard  for 


the  first  fi/e  studies,  which  were  designed  tu  examine  the  form  cf  objectives. 

Study  1  supplied  us  with  normative  data  regarding  the  "cbservaoil ity ' 
of  99  ''reqeenth,  recommenced  'ebs,  a  replicator.  of  Dene  jnc  denk'ns  (1959;. 


The  results  confirmed  their  findings,  but  carried  with  them,  as  had  previous 
studies,  the  Indication  that  the  verb  Is  the  only  part  of  the  objective 
worth  considering. 

Study  2  began  an  examination  of  the  effect  of  other  parts  of  the 
objective,  l.e.,  conditions  and/or  criteria,  on  the  already  established 
"observability"  of  the  verbs  In  Study  1.  Both  components  were  found  to 
affect  the  nature  of  the  objective  as  well  as  the  perceived  precision  of 
the  verb,  often  In  a  highly  significant  manner. 

Study  3  further  developed  the  role  of  the  three  components  of  perform¬ 
ance  objectives.  These  data  suggested  that  the  observed  precision  of  an 
objective  results  from  an  Interaction  of  all  thre$  parts,  and  that  analyses 
of  the  components  In  Isolation  are  meaningless. 

The  search  for  unaccounted-for  variance  In  the  overall  observability 
of  objectives  led  to  an  examination  of  yet  another  segment,  the  direct 
object.  The  objectives  employed  In  Stud  as  1  through  3  used  an  abstract 
direct  object  ("x"  or  "y")  In  order  to  minimize  differential  effects.  In  Study  4, 
direct  objects  varying  In  judged  degree  o  specificity  were  incorporated  into 
the  objectives  and  tested  for  their  specific  and  general  effects.  The  direct 
objects  were  found  to  contribute  important  additional  Information  to  the 
reader  In  terms  of  both  observability  and  precision,  following  the  linear 
relation  anticipated  by  the  gradations  In  specificity.  Interestingly, 
comparisons  of  objectives  with  either  abstract  or  concrete  direct  objects 
produced  no  overall  significant  difference,  suggesting  that  readers  had 
naturally  substituted  concrete  modifiers  for  the  abstract  direct  objects  In 
order  to  "complete"  the  objective.  An  acceptable  performance  objective, 
therefore,  should  contain  not  only  a  verb  denoting  an  observable  behavior, 
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researcher  did  take  great  pains  to  provide  a  comprehensive  training,  there 
would  be  no  guarantee  that  the  comparison  group  would  not  have  received 
similar  "objective-based  training"  by  conventional  means. 

Epilogue:  When  it  doesn't  work  out  empirically,  rationalize 

To  continue  elaborating  on  the  above  would  force  us  to  go  beyond  the 
data.  Thus,  clearly  demarcated,  the  following  brief  analysis  is  a  rational 
defense  for  the  use  of  performance  objectives. 

The  complaints  which  reasonably  well -motivated  students  most  often 
make  about  courses  fall  Into  three  main  categories:  (a)  they  do  not  under¬ 
stand  what  Is  going  on,  (b)  they  perceive  the  course  as  being  too  much  work, 
and/or  (c)  there  are  components  in  the  instructional  environment  which  are 
either  frustrating  or  unjust.  Objectives  provide  a  direct  and  positive 
means  for  reducing  or  eliminating  all  three.  The  use  of  objective-based 
training,  or  the  systems  approach,  automatically  excludes  the  first  complaint 
because  the  student  either  lacks  the  necessary  prerequisite  knowledge  and 
is  denied  admission  until  this  deficiency  is  removed,  or  the  "objectives" 
themselves  are  ambiguous  or  Incomplete,  a  circumstance  wnich  the  instructor 
must  rectify.  The  systems  approach  also  addresses  the  second  class  of 
complaints:  all  good  Instruction  is  learner-paced,  or  the  instructional 
system  must  be  adaptive  to  the  learner.  In  addition,  objective-based 
instruction  tends  to  be  simpler  (though  we  have  no  empirical  evidence  for 
this  assertion),  a  pnenomonen  which  may  be  a  byproduct  of  clearer  thinking. 
Furthermore,  this  "simple"  Quality  would  obscure  sought-afte1"  differences 
in  empirical  research.  Finally,  performance  objectives  Help  eliminate 


frustration  and  unfairness  because  of  (a)  reduced  test  threat,  (b)  apparent 
concern  for  the  student  by  the  Instructor  (affective),  (c)  determination 
of  a  starting  point,  content,  and  a  realizable  end,  and  (d)  equal  requirements 
(explicitly  stated)  for  all. 

The  question  remains,  "Do  objectives  facilitate  learning?"  The 
rational  response  would  be,  "...more  research  Is  not  needed  to  establish 
a  clear  positive  relation  between  objectives  and  good  Instruction."  However, 
both  rational  thought  and  the  empirical  evidence  presented  In  this  report 
suggest  that  work  Is  needed  In  the  area  of  teacher  rr  Instructor  training 
to  promote  the  behavior'  Induced  by  the  effective  use  of  performance  objectives. 
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This  is  a  study  designed  to  determine  the  extent  to  which  various  words  are 
labels  for  behavior  which  is  directly  observable.  For  example,  most  would  agree 
that  the  verb  "to  hit"  labels  behavior  which  you  can  see,  while  the  verb  "to 
believe"  labels  an  internal  state  which  cannot  be  directly  observed. 

On  the  subsequent  pages  you  will  find  e  list  of  action  words,  or  verbs. 

Your  task  Is  to  rata  each  word  on  a  scale  from  1-5  from  most  observable  tc  least 


observable  as  follows: 

Most  Observable 

least  Observable 

1  2 

3 

4  3 

to  hit 

to  believe 

to  bite 

to  sympathize 

Words  such  as  "to  ait"  and  "to  bite"  are  to  be  given  a  rating  of  1,  while 
words  such  as  "to  believe"  and  "to  sympathize"  are  rated  as  5. 

Many  of  the  words  nay  not  in  your  judgment  be  rated  1  or  5,  and  these  you 
are  to  rate  as  2,  3,  or  4  as  you  see  fit. 

Remember,  the  rating  you  give  le  determined  by  the  extent  to  which  you 
judge  it  possible  to  observe  the  behavior. 

PLEASE  PUCE  IRE  NUMBER  WdICH  YOU  GIVE  AS  A  RATING  IN  FRONT  OF  THE  WORD. 


Please  write  the  number  of  the  month  and  data  of  your  birth,  plus  .  .at 
two  digits  of  your  Social  Security  nuvbar,  on  the  line  below: 


Month  Date  S.S.  # 

(Last  2  digits) 

This  six  digit  code  preserves  your  anonymity.  It  also  enables  us  to  pair 
results  of  today's  questionnaire  with  e  later  one,  should  thst  become  necessary. 


Thank  you! 


'  f  -■ .  on 


_ to  staie 

•-  eet. 
to  vest 
....  r.o  utilize 

_  V'  c.-der 

.. . .  „  * 1  ‘  &  r  o  no 

fy  understand 

... _ t l  hank 

to  sol^e 

_  to  seiner 

_ _  to  recognize 

_ _  to  round  off 

_  to  shade 

_  t  o  Gunzaarlza 

_  to  total 

. _  to  use 

_  ft  like 

_ ._  to  walk 

_  to  tell  what 

_ to  think  or  ideally 

_  to  subtract 

_  to  realize  fully 

_  to  regroup 

_ _ _  to  say 

_ _  to  supply 


o  ux.ae:  .  f  \ 
j  vr . r * 


vender 


to  learn 
read 


to  take  awav 


_  to  add 
_  to  analyse 
_  to  ba  curious 
.  to  count  orally 
.  to  change 
.  to  concentrate 
to  develop 
to  determine 
to  draw 
to  complete 
to  check 


_  to  appreciate 
_  to  arrange 

_  to  apply 

-  t0  become  competent 
_  to  circle 
.  to  choose 
.  to  evaluate 
,  to  dlatlngylah 
to  discriminate 
to  divide 
to  convert 
to  construct 
to  cover  vith  a  card 
to  borrow 
to  average 


.  to  acknowledge 
.  to  cross  out 
to  connect 
to  create 
to  demonstrate 
to  discover 
to  be  aware 


to  deduce 


to  label 


to  give 
to  identify 
co  locate 
to  multiply 
to  place 
to  read  orally 
to  perform 
to  point  to 
to  make 
to  line-draw 
to  inquire 
to  finish 
to  find 

to  identify  in  writing 

to  lower  press 

to  match 

to  name 

to  partition 

to  remove 

to  play 

to  number 

to  mark 

to  know 

to  generate 

to  feel 

to  fill  in 


to  measure 
to  put  on 
to  perceive 
to  repeat  orally 


to  infer 


/ 
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